Link to Article

The rapid advancement of AI algorithms and open-source datasets has revolutionized various industries, including architecture (Hovestadt, Hirscheberg, and Fritz 2020; Chaillou, 2021). Open-source datasets provide a vast amount of information that enhances architectural projects, while big data plays a vital role in training Generative Algorithms (GA) and optimizing Search Engine (SE) indexing.

This article presents a novel architecture design workflow that explores the intersection of Big Data, Artificial Intelligence (AI), and storytelling by scraping, encoding, and mapping data, which can then be implemented through Virtual Reality (VR) and Augmented Reality (AR) technologies. In contrast to conventional approaches that consider AI solely as an optimization tool, this workflow embraces AI as an instrument for critical thinking and idea generation. Rather than creating new AI models, this workflow encourages architects to experiment with existing ones as part of their practice.

This article presents a framework that incorporates search engines into architectural design processes, advocating against starting from scratch and, instead, leveraging preexisting loaded data and information, including text, images, 3D models, and more. These conceptualizations can be further explored using traditional 3D software or experimented with in AR/VR platforms. Through this approach, the framework aims to cultivate a sensibility for working with vast amounts of data, while maintaining focus and effectively utilizing the available online resources to articulate architectural projects.

As a proof-of-concept, this methodology was applied in a 6-week-long workshop involving architecture students.

METHODOLOGY: ARTICULATING JOINTS IN MANY DIMENSIONS
The workflow is organized into three modules: Mapping Big Data, Collapsing Dimensions, and Staging Multi-dimensional Stories. These modules were applied to the initial conceptual stage in the design process.

• Mapping Big Data
  Accessing the plenty

The workshop utilized AI algorithms to crawl the internet for information about sites and users. Data were collected in two modalities - Images and 3D Models.

Images: To gather relevant data for our analysis, we employed an automated scraping tool specifically designed to extract social media posts (Figure 1). This tool allowed us to collect both images and text based on specified keywords, time ranges, and location criteria. The primary objective was to conduct a comprehensive site analysis and gain insights into user needs within a specific geographical context. This approach allows us to gather data encompassing various dimensions, such as sentiment (based on keywords), time (based on a specified time range), and geography (based on location criteria).

Models: We collected 3D models of canonical architecture, a total of 38 projects, mostly at the scale of small houses. These projects were drawn primarily from two publications curated by the authors as key projects of the 20th century (Davies 2006; Weston 2004). Moreover, the supplied CAD drawings gave a relatively precise foundation on which to reconstruct the 3D models of these projects (Figure 2).

• Dimensional Transversality

The collected data was preprocessed by cleaning and optimizing the 3D data and image resizing for 2D data. We also used AI algorithms, such as autoencoders (Simonyan and Zisserman 2015) and Fourier transform (Bracewell 1965) to extract feature vectors and encode their modality into numerical representations.

Images: We used a pre-trained Convolutional Neural Network (CNN) called VGG16 (Simonyan and Zisserman 2015) to extract feature vectors from the images. A trained VGG16 iteratively conducts non-linear operations on the input image of 224 x 224 x 3 pixels to reduce the size of the input image by each operation and eventually converts the input image to a 1,000-dimensional vector that represents the probability of the input image being each of the 1,000 predefined categories of objects.

Models: The models were built from scratch or refined into clean Non-Uniform Rational B-Splines (NURBS) geometry as a preparatory step. Each model was segmented into a matrix of eight-foot cubes using recursive boolean operations with a Rhino Grasshopper definition. Depending on project size, this would yield anywhere from 60 to 800 segmented cubes with embedded architectural elements (Figure 3).

• Design Space Representation

After encoding the data modalities, we employed the Self Organizing Maps (SOM) algorithm (Kohonen 1982) to serve as our search engine for both types of data. The SOM performs the crucial task of visualizing the collected data in a condensed space, enhancing our ability to navigate and comprehend vast amounts of information. By utilizing SOM, we can transform high-dimensional data into a lower-dimensional space or “maps”, while preserving the original topology.

Maps Images: Using the SOM on the feature vectors of the collected images, the output map curates a selection of images organized/clustered based on their feature vector similarities (Figure 4). This map was used to create atmospheric collages indexing time and space (Figure 5). Since the data was collected using geo coordinates and posting time, we could create images representing these aspects from various personal viewpoints.

Maps Models: Using the SOM in the feature vectors of the 3D models, we were able to create a subsample of selected details, as similar details were clustered together (Figure 6). Since the eight-foot cube dimension always captures a substantial building element (floor/ceiling/facade/circulation), distinct features with unique elements stand out, while generic ones are filtered out. These are assessed for desirable properties, and utilized as seeds for the design proposals (Figure 7).

• Collapsing Dimensions

Participants used the Oculus Quest 2 and Gravity Sketch, a VR modeling program, to explore the potential of body-scale modeling to generate derivatives of selected canonical building details, including the created atmospheric images (Gravity Sketch Ltd. 2022). Referencing the SOM of 3D details created previously, participants chose details that embodied specific spatial properties, and arranged them loosely in an exquisite-corpse cadaver exquis (Brotchie and Gooding 1991) style arrangement, taking into account site, scale, and relative spacing between these fragments to correspond with expected programmatic ideas and requirements. These were imported into Gravity Sketch to be simultaneously experienced and manipulated at 1:1 scale in VR. This sets up a situation where the 3D details helped bracket very specific spatial conditions, while the interstitial space remained for participants to bridge, interpret, and morph. Simultaneous collective work between participants occupying the shared VR workspace across scales led to an exquisite corpse-style assemblage of proposals that occupied the site with sensitivity to both the detail scale internally, as well as the urban scale externally (Figure 8).

• Staging Multi-Dimensional Stories

The previous work (atmospheric images and exquisite corpse-style assemblage) produced with this workflow was placed within the environment of the game engine Unity (Unity Technologies 2023) to crystallize architectural proposals. Game engines are media agnostic and can deploy any media; text, images, movies, animations, drawings, renders; and spaces, thereby finding their coherence, not in the specificity of the media or geometries, but in the grounding of the narratives. A scenographic setting was composed by combining many media that actively discuss and construct architectural proposals through scale, context, elements, and details, living in multiple worlds at the same time.

The previously mapped images and models, and VR sketches, are placed within a rendering environment where the post-processing takes precedence over geometric modeling (Figure 9). Always focused on the narrative layer of the project, game engines deal with objects that are always active, communicating asynchronously with each other within a space that is also always alive.

The proposed workflow introduced an approach to integrating AI and storytelling technologies into architectural design, revolutionizing a traditional process. Leveraging AI algorithms for data collection, processing, and visualization enables architects to gather and analyze vast amounts of data, providing deeper insights into user needs, and urban and spatial issues. By utilizing VR-based modeling and collaborative virtual workspaces, participants engage in an interactive and creative design process that fosters innovation and produces unique proposals. The application of AI search engine techniques, instead of generative algorithms serves as analytical, conceptual, and geometric tools that complement human interpretation, emphasizing the designer's intent, rather than subsuming human creativity under the machine.

This data-driven approach, combined with real-time storytelling techniques and VR technologies, offers a multi-scalar platform for ideating and developing architectural proposals, bridging the gap between n-dimensional connections and tangible representations.