Demo¶

Xunpu¶

Image process¶

数据来源：微博搜索关键词：浔埔，潯埔，潯埔（村）数据选取原则：

高清图
水印可裁切
背景简单 head wreath head flower

扣背景

Evaluate¶

泛化性

../_images/xyz_grid-0029-2837918259.png — LoRA train with/without regularized data for anything v5¶

xunpu, (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair
Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2837918259, Size: 512x768, Model hash: e06702b274, Model: AnythingV5Ink_v5RE, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_noreg(3dc56c73994a), AddNet Weight A 1: 0.1, AddNet Weight B 1: 0.1, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: 0.1-1.1(+0.2), Y Type: AddNet Model 1, Y Values: "xunpu_noreg(3dc56c73994a),xunpu_reg(f55d1424cb0b)", Version: v1.3.2

../_images/xyz_grid-0031-2837918259.png — LoRA train with/without regularized data for difference ckpt¶

xunpu, (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair
Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2837918259, Size: 512x768, Model hash: e06702b274, Model: AnythingV5Ink_v5RE, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_noreg(3dc56c73994a), AddNet Weight A 1: 0.8, AddNet Weight B 1: 0.8, Script: X/Y/Z plot, X Type: Checkpoint name, X Values: "AnythingV5Ink_v5RE.ckpt [e06702b274],beautifulRealistic_brav5.safetensors [ac68270450]", Y Type: AddNet Model 1, Y Values: "xunpu_noreg(3dc56c73994a),xunpu_reg(f55d1424cb0b)", Version: v1.3.2

正则化数据的影响¶

../_images/xyz_grid-0025.png — LoRA train with/without regularized data for difference seeds¶

不同种子对结果的影响

xunpu, (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed,cute, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair

Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 0, Size: 512x768, Model hash: ac68270450, Model: beautifulRealistic_brav5, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_noreg(3dc56c73994a), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Script: X/Y/Z plot, X Type: Seed, X Values: "0,1,2,3,4,5,6", Fixed X Values: "0, 1, 2, 3, 4, 5, 6", Y Type: AddNet Model 1, Y Values: "xunpu_noreg(3dc56c73994a),xunpu_reg(f55d1424cb0b)", Version: v1.3.2

../_images/xyz_grid-0007-2837918259.png — LoRA train with/without regularized data¶

1 girl, (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed,cute, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair

Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2837918259, Size: 512x768, Model hash: ac68270450, Model: beautifulRealistic_brav5, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_noreg(3dc56c73994a), AddNet Weight A 1: 0.1, AddNet Weight B 1: 0.1, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: 0.1-1.1(+0.2), Y Type: AddNet Model 1, Y Values: "xunpu_noreg(3dc56c73994a),xunpu_reg(f55d1424cb0b)", Version: v1.3.2

过拟合

../_images/xyz_grid-0018-2837918259.png — prompts overfitting of LoRA with/without regularized data¶

1 girl, (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed,cute, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair

Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2837918259, Size: 512x768, Model hash: ac68270450, Model: beautifulRealistic_brav5, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_noreg(3dc56c73994a), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Script: X/Y/Z plot, X Type: Prompt S/R, X Values: "1 girl, xunpu", Y Type: AddNet Model 1, Y Values: "xunpu_noreg(3dc56c73994a),xunpu_reg(f55d1424cb0b)", Version: v1.3.2

../_images/xyz_grid-0023-2837918259.png — prompts overfitting of LoRA with/without regularized data¶

xunpu, (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed,cute, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair

Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2837918259, Size: 512x768, Model hash: ac68270450, Model: beautifulRealistic_brav5, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_noreg(3dc56c73994a), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Script: X/Y/Z plot, X Type: Prompt S/R, X Values: "white shirt, red shirt, chinese shirt, blue sweater", Y Type: AddNet Model 1, Y Values: "xunpu_noreg(3dc56c73994a),xunpu_reg(f55d1424cb0b)", Version: v1.3.2

../_images/prompt_matrix-0007-2837918259.png — prompts matrix LoRA without regularized data¶

../_images/prompt_matrix-0006-2837918259.png — prompts matrix LoRA with regularized data¶

(8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair, from side

Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2837918259.0, Size: 512x768, Model hash: ac68270450, Model: beautifulRealistic_brav5, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_reg(f55d1424cb0b), AddNet Weight A 1: 0.6, AddNet Weight B 1: 0.6, Version: v1.3.2

zenli¶

Image process¶

数据来源：曾黎的微博相册2020年至今数据选取原则：

一致性
多样性
高清
主体明确

数据处理：

裁切重点区域，脸部
涂抹，填充复杂背景区域，文本区域，水印
过大需要缩放

make caption and tag¶

blip wd14

$ cd sd-scripts
$ python finetune/make_captions.py /code/data --caption_extension .cap --batch_size 4 --max_data_loader_n_workers 4 --recursive
$ python finetune/tag_images_by_wd14_tagger.py /code/data --thresh 0.5 --caption_extension .tag --batch_size 4 --max_data_loader_n_workers 4

Training¶

Install sd-scripts¶

¶

Evaluate¶

../_images/xyz_grid-0032-2837918259.png — zenli and xunpu weight matrix¶

../_images/xyz_grid-0033-2837918259.png — zenli and xunpu weight matrix¶

xunpu, zenli, (8k, RAW photo, best quality, masterpiece:1.2), (realistic, photo-realistic:1.37), ultra-detailed, solo, simple background,beautiful detailed eyes,(white shirt:1.1),(short hair:1.2),floating hair
Negative prompt: (face mole:1.3), (hands:1.4), EasyNegative, paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,(watermark:1.2),(white letters:1/1)
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2837918259, Size: 512x768, Model hash: ac68270450, Model: beautifulRealistic_brav5, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: xunpu_reg(f55d1424cb0b), AddNet Weight A 1: 0.1, AddNet Weight B 1: 0.1, AddNet Module 2: LoRA, AddNet Model 2: zenli(44709aece392), AddNet Weight A 2: 0.1, AddNet Weight B 2: 0.1, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: 0.1-1.1(+0.2), Y Type: AddNet Weight 2, Y Values: 0.1-1.1(+0.2), Version: v1.3.2