So I'm making a project for a school that offers image generation using stable diffusion. It was working perfectly fine until I upgraded the Pytorch version for "stabilityai/stable-diffusion-x4-upscaler" model. Since then all the other images generated are black because they get flagged NSFW.
The code hasn't changed but I have updated Mac OS to 13.3.1 recently and I tested many different versions of PyTorch, Diffusers, and Transformers, but I can't get it to work as it did before. Since the code didn't change the problem has to do with the package version.
With the latest PyTorch 2.0 I am able to generate working images but I cannot use torch_dtype=torch.float16 in the pipeline since it's not supported and I seem to be getting the following insufficient memory issues now.
RuntimeError: MPS backend out of memory (MPS allocated: 18.04 GB, other allocations: 94.99 MB, max allowed: 18.13 GB). Tried to allocate 4.00 KB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure). INFO: Stopping reloader process [15702]
These are the following models and piplines I used.
main_model_id = "runwayml/stable-diffusion-v1-5"
inpainting_model_id = "runwayml/stable-diffusion-inpainting"
upscaler_model_id = "stabilityai/stable-diffusion-x4-upscaler"
text2imgPipe = StableDiffusionPipeline.from_pretrained(main_model_id, torch_dtype=torch.float16).to(
device)
text2imgPipe.enable_attention_slicing()
img2imgPipe = StableDiffusionImg2ImgPipeline.from_pretrained(main_model_id, torch_dtype=torch.float16).to(
device)
img2imgPipe.enable_attention_slicing()
inpaintingPipe = StableDiffusionInpaintPipeline.from_pretrained(inpainting_model_id, torch_dtype=torch.float16).to(
device)
inpaintingPipe.enable_attention_slicing()
upscalerPipe = StableDiffusionUpscalePipeline.from_pretrained(upscaler_model_id, torch_dtype=torch.float16).to(
device)
upscalerPipe.enable_attention_slicing()
These are the current package version I have.
transformers 4.26.0
torchaudio 2.0.0
torchvision 0.14.1
pytorch 2.0.0
diffusers 0.12.0
huggingface-hub 0.13.4
python 3.10.10
