Kirazuri Lazuli (Noobai V-Pred)

This checkpoint is a personal project trained locally from NoobAI-XL (NAI-XL) V-Pred 1.0-Version on a 4090 over one month on a small dataset of 12,561 total images.

It focuses on adding additional knowledge since the data cutoff of the base model (2024/10/24), including styles, concepts, and characters from anime, video games, and virtual youtubers.

As a small personally curated dataset, please be aware this is not inclusive of everything released since that cutoff

Due to the training approach used, the styles, concepts, and characters included towards the end of the training are learned better
Training the text encoders when introducing novel character trigger tokens, and using a well structured regularization dataset are things I learned are important

Usage

Your preferred generation settings used for NoobAI-XL (NAI-XL) V-Pred 1.0-Version should be mostly transferrable.

Previews are generated with a ComfyUI workflow using DynamicThresholdingFull, Upscaling, and FaceDetailer.

DynamicThresholding (CFG-Fix) settings used with a CFG of 10:

dynthres_enabled: True, dynthres_mimic_scale: 7, dynthres_threshold_percentile: 1, dynthres_mimic_mode: Half Cosine Down, dynthres_mimic_scale_min: 1, dynthres_cfg_mode: Half Cosine Down, dynthres_cfg_scale_min: 3, dynthres_sched_val: 1, dynthres_separate_feature_channels: enable, dynthres_scaling_startpoint: ZERO, dynthres_variability_measure: STD, dynthres_interpolate_phi: 1

For samplers, recommendation is Euler for generation, Euler Ancestral for upscaling/inpainting.

reForge or Forge should also be usable as resolved from version 1.0 (apologies if you ran into issues with that version).

*To be automatically detected as a v-pred model in Forge/reForge, I manually added znstr and v_pred keys to the state dict of the model using this script.

Recommended prompt structure:

Quality modifiers masterpiece, best quality, very aesthetic should be positioned at the end of the prompt.

Artist names can be prefixed with artist: to prevent token bleeding with artist names and concepts.

A1111 schedule prompting syntax is used in ComfyUI through the comfyui-prompt-control extension to combine artist styles, i.e: artist:[artist1|artist2|artist3]

In some cases Regional Prompting with Attention Couple is also used.

Positive prompt:

{{characters}}, {{copywrites}}, {{artists}}, {{tags}}, absurdres, masterpiece, best quality, very aesthetic

Training details

The kohya-ss/sd-scripts training configs used can be found on github.

Iterative checkpoint training approach inspired by PixelWave.

This involved training in dataset batches of ~1200 images, for 10 training sessions, before finishing with an 11th aesthetic finetune dataset of 267 images.

Adafactor optimizer
Full-fat fp32 training precision
Batch size and LR were adjusted multiple times
- Batch size 4, LR 6e-6 seemed most stable
TE trained for the 10th and 11th training sessions at Batch size 4, LR 2e-6
Regularization dataset generated from the 10th checkpoint used in the final aesthetic training to preserve the previously learned characters

Recognitions

Thanks to Laxhar Lab for the NoobAI-XL (NAI-XL) V-Pred 1.0-Version base model.

Thanks to narugo1992 and the deepghs team for open-sourcing various training sets, image processing tools, and models.

Thanks to kohya-ss for the sd-scripts trainer.

License

This model's license inherits from https://huggingface.co/OnomaAIResearch/Illustrious-xl-early-release-v0 fair-ai-public-license-1.0-sd and adds the following terms. Any use of this model and its variants is bound by this license.

I. Usage Restrictions

Prohibited use for harmful, malicious, or illegal activities, including but not limited to harassment, threats, and spreading misinformation.
Prohibited generation of unethical or offensive content.
Prohibited violation of laws and regulations in the user's jurisdiction.

II. Commercial Prohibition

We prohibit any form of commercialization, including but not limited to monetization or commercial use of the model, derivative models, or model-generated products.

III. Open Source Community

To foster a thriving open-source community, users MUST comply with the following requirements:

Open source derivative models, merged models, LoRAs, and products based on the above models.
Share work details such as synthesis formulas, prompts, and workflows.
Follow the fair-ai-public-license to ensure derivative works remain open source.

IV. Disclaimer

Generated models may produce unexpected or harmful outputs. Users must assume all risks and potential consequences of usage.

Kirazuri Lazuli (Noobai V-Pred) - v1.1