Member-only story
Train Your Own Stable Diffusion Model Locally — No Code Needed
I’ve been playing around with Stable Diffusion for some weeks now. Started with the basics, running the base model on HuggingFace, testing different prompts. Then I started reading tips and tricks, joined several Discord servers, and then went full hands-on to train and fine-tuning my own models. Now I have a model made with my own face, and I can create custom avatars and whatnot. It has been fun.
Last week I was talking to a colleague and he told me that he was really interested on generative AI, but unfortunately lacked the technical knowledge needed to train a custom model, so was just running inference on the base model. And I suddenly thought — what if I make something for him to play with, no code needed at all? Something like “hey man, just run this script — it will create the model you want, based on your input images”.
Sounded like an interesting project, so I started researching how to do it. And, well, I was late. Actually, I was several months late. Everything I wanted to build already existed, made by people really smart that are lightyears ahead of me in terms of AI exposure and experience. So I humbly forgot about my project and wrote a tiny tutorial to my friend instead, explaining how to train your model.
This is the extended version.
Requirements — and full disclosure
Although this should work fine with any OS, since all the repositories and web UIs we’re going to use have support for Linux, Windows, and Mac, I have only tested this with my own and my friends’ setup:
- Windows 10
- GPU with at least 6–7GB of VRAM — more precisely, this works fine with an NVIDIA GeForce RTX 3060 or a RTX 3060 Ti. That is not enough to run pure Dreambooth, but we’ll use LoRA so it’s fine — don’t worry if you don’t understand this yet, we’ll come back later. I believe some of the features can run on the CPU only, but I haven’t tested that.
Besides that, you will need to install two things:
- Python 3.10. Be sure to check the options “Add Python to PATH” and “Support for TCL/Tk”.
- Git
