From Zero to Crowd: A Guide to 3D Crowd Generation using Stable-Diffusion and Blender

Exploring automated 3D crowd generation methods, limitations, and potential improvements

Blender Results Demo

This article provides an overview of the process for generating diverse and realistic 3D crowds. We will begin by examining the current simplest approach, as well as its limitations, and explore possible paths for improvement. The process is broken down into three steps:

1. generating 2D images
2. reconstructing 3D objects
3. Importing and scattering objects in Blender to simulate a crowd

Generate 2D Images

The goal of this step is to create a rich, diverse, and realistic image set of full-body characters that will make up our crowd.

Generative-models are the machine-learning technique needed for this task, and while in the past GANs were the undisputed choice, now is all about diffusion-models. The most notable names are DALL·E 2, Midjourney and Stable-Diffusion.

We choose Stable-Diffusion as it is fully open-source, with an extended ecosystem of tools and a vibrant community. We also rely on (and recommend) the Automatic1111 WebUI to generate the images. There are a lot of…