BTW
@Kadayi just for better understanding SD: would it be correct to say that checkpoints and loras are basically lower instances of the internet? In ozher words, an Ai that accesses the internet has the most data to pull from. A checkpoint is a large collection of data that can create a lot, but not everything. And a Lora is a small collection of data tailor-made for a specific task. Correct?
You need a checkpoint always as that is your base model. Checkpoints are trained on vast amounts of image data.
LoRAs are effectively trained mini-models of a particular subject or style that effectively interface with the main Checkpoint model, these are usually from 8 - 200mb in size dependent on the training.
On top of
Checkpoints &
Lora you also have
Embeddings, which are generally quite small files (normally sub 1mb) and oftentimes are codified collections of negative prompts, or people's likenesses.
Let's take the image that I posted earlier on. This was generated using the
Rev Animated model which is quite a popular stylised model.
For the sake of expediency to illustrate the inpainting issue I just used a simple prompt that I cribbed from Civitai: -
Prompt: pop art, 2 Girls, Scarlett Johansson and Kirsten Dunst, upper body, hair, red lips, looking at viewer, hat, <lora: pop_art_v2:0.7>
^ as you can see the Prompt has a style
Lora reference to it in between the < > and there is a weight to it of 0.7 (generally so as not to overpower the checkpoint you want to go for 0.3 - 0.9) , and "pop art" is the associated trigger word
Negative prompt: EasyNegativeV2 ng_deepnegative_v1_75t bad_prompt_version2
^These are all Embeddings, which you should be able to find quite easily using search at civitai. Just install them to your Embeddings folder.
Steps: 20, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 874390029, Size: 768x768, Model hash: f8bb2922e1, Model: revAnimated_v122, VAE hash: 551eac7037, VAE: vae-ft-mse-840000-ema-pruned.ckpt, Clip skip: 2, Token merging ratio: 0.2,
^ these are the image settings plus the image seed. Using the same prompt, plus seed you should end up with a very similar image (although that can vary depending on your SD settings).
Lora can be quite good, esp for things like artistic styles, or people, however, they can overpower a model somewhat and you need to balance that out in terms of the weight. Embeddings are pretty good for likenesses and operate off of keywords usually something like 3mmaston3 for instance.
Same prompt as before save 1 girl using Rev Animated checkpoint and the same seed but using embeds for Alison Brie, Emma Stone, Anya Taylor-Joy respectively
Don't bother with Hypernetworks, as they are barely used. LyCoris and Lora are interchangeable and are effectively the same thing and operate in the same way. LyCoris is a newer tech though and generally, the files sizes are a bit smaller compared to Lora.