2

I am tweaking a python script using diffusers inpainting pipeline for a custom video generation idea.

I would like to gradually shift the weights of certain words in the prompt.

As I understand the argument prompt_embeds is exactly what i need.

I could not figure out how to define this argument. Can someone pls provide an example?

I know there are frameworks out there where you can just add weights to certain words with the following syntax:

"This is a SD prompt with plus 50% weight added to the last (word:1.5)"

This would be a great solution as well however this does not work with diffusers.StableDiffusionInpaintPipeline

Bálint Komjáti
  • 33
  • 4

1 Answers1

1

Example from the original repo of Compel library:

from diffusers import StableDiffusionPipeline
from compel import Compel

pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder)

# upweight "ball"
prompt = "a cat playing with a ball++ in the forest"
conditioning = compel.build_conditioning_tensor(prompt)
# or: conditioning = compel([prompt])

# generate image
images = pipeline(prompt_embeds=conditioning, num_inference_steps=20).images
images[0].save("image.jpg")

So, as you can see, you should add ++, or --, or ----- and so on. For your case it would be like "This is a SD prompt with plus 50% weight added to the last word+++++". Also you can up\downweight subprompt: A rich and descriptive (a subprompt goes here)++++ prompt.

Alex Glinsky
  • 3,346
  • 1
  • 17
  • 14