A combined character lora using Amy Wong from Futurama , Leelah from Futurama , Lisa simpson from the simpsons , Jessica Lovejoy from

the Simpsons , Allison Taylor from the Simpsons , Terri / Sherri Mackleberry from the Simpsons.

All training images are illustrative in style of 'The Simpsons' and 'Futurama' tv shows.

The captions were manually appended with a character name specific string , that list the names of the characters appearing in the training image from left to right.

For example 'Amy Wong to the left , then Jessica Lovejoy , then Lisa Simpson and then Amy Wong to the right' . It might be enough to train character name in this LoRa. See if it works.

Recommended prompt format:

Use a combination of joycaptions and danbooru style prompts.

Many characters shots were processed using Qwen Edit AIO to enhance vibrancy of colors , and make the background dark gray. The output resolution of the image edits are 1024x1024 or 512x1024 at cost of 0.2 tokens per image. Total cost roughly 50 tokens after processing probably 100 images this way. But it was off the daily tokens budget so no harm done :) !

After manually trimming off each training image horizontally , mainintaining the 1024px resolution height , a google colab script was used that placed all the 200+ training images on a single row randomly , from which N 1024x1024 square images were 'cut' to produce the LoRa training frames , upon the best ones were selected creating an image training set of 140 frames at 1024x1024 resolution.

Captioning of images was done using joycaptions , and then the dataset was tagged using dino tagger. When mixing the the tags from dino tagger and the sentences from joycaptions , the colab script was tasked to write the joycaptions sentences for each caption in random order , and place the tags in random order between the sentences. For 8 sentences joycaption prompt , this put all the dino tags in random order in the middle after 4 sentences , which wasn't my intention but good enough.

The reason for randomizing the caption sententence and tag order is to mimic the rotary position encoding system used to train models like klein in this lora training session.

Training parameters:

Learning rate 0.00035 using 'cosine with repeats' for rank 32 and alpha 32.

Using 3 repeats x 40 epochs x 140 images = 17200 training steps

Using 50 warmup steps. No gradient accumulation. Learning rate scheduler step of 3 , so three 'golf swing strengths' 0.00035 -> 0.0002 -> 0.0001 -> 0.00035 -> ... etc.

Simpsons Style - e38