Auto shrink label messes up paragraph spacing

Question

I have an attributed string in a UILabel. The problem is if I want the UILabel to auto shrink the text then the custom paragraph spacing is messed up. Please see this picture for an example:

Design

This is my goal in terms of layout:

Code

To get auto shrink to work on the UILabel, I do this from this answer:

let label = UILabel()
label.adjustsFontSizeToFitWidth = true
label.attributedText = attributedString

// Auto shrink only works if the lineBreakMode is set after 
// setting the attributed string. It seems like the attributed 
// string overrides line break behavior when set (among other things).
label.lineBreakMode = .byTruncatingTail

The problem is that now this does not respect paragraph spacing. Everything is close together. This ignores the spacing I set for paragraphSpacingBefore and paragraphSpacing.

This is the full code for my string:

static func attributedCustomString(topText: String, bottomText: String) -> NSMutableAttributedString {
    let fullString = NSMutableAttributedString()
    
    let mainStringParagraphStyle = NSMutableParagraphStyle()
    mainStringParagraphStyle.alignment = .center
    mainStringParagraphStyle.lineHeightMultiple = 1.4 // This is a multiplier, not a value in points
    
    let mainString = NSAttributedString(string: "\(topText)\n",
                                        attributes: [.paragraphStyle: mainStringParagraphStyle,
                                                     .font: UIFont.systemFont(ofSize: 28)])
    
    fullString.append(mainString)
    
    let lineImageStringParagraphStyle = NSMutableParagraphStyle()
    lineImageStringParagraphStyle.alignment = .center
    lineImageStringParagraphStyle.paragraphSpacingBefore = 24.0 // The space before image
    lineImageStringParagraphStyle.paragraphSpacing = 8.8 // The space after image
    
    let lineImageAttachment = NSTextAttachment(image: #imageLiteral(resourceName: "line-separator"))
    let lineImageString = NSMutableAttributedString(attachment: lineImageAttachment)
    lineImageString.addAttribute(.paragraphStyle,
                                 value: lineImageStringParagraphStyle,
                                 range: NSRange(location: 0, length: lineImageString.length))
    
    let bottomStringParagraphStyle = NSMutableParagraphStyle()
    bottomStringParagraphStyle.alignment = .center
    bottomStringParagraphStyle.lineSpacing = 1.4
    
    let bottomString = NSMutableAttributedString(string: "\n\(bottomText.uppercased())",
                                                 attributes: [.paragraphStyle: bottomStringParagraphStyle,
                                                              .font: UIFont.systemFont(ofSize: 14),
                                                              .foregroundColor: UIColor.white.withAlphaComponent(0.9)])
    bottomString.addAttribute(NSAttributedString.Key.kern, value: 5.0, range: NSRange(location: 0, length: bottomString.length - 1))
    
    fullString.append(lineImageString)
    fullString.append(bottomString)
    
    return fullString
}

Any thoughts on how to make this work?