Change weight and color of System Symbol UIImage in Swift

Question

I'm trying to change the weight, scale, and color of the system symbol since that isn't an option to do all this in one go in the UIImage.SymbolConfiguration I've tried to do this in an extra step. The color doesn't apply but the SymbolConfiguration do so now I'm wondering what I am doing wrong.

Code:

    let backButtonImage: UIImage = {
        var config = UIImage.SymbolConfiguration(pointSize: 24, weight: .heavy, scale: .large)
        let image = UIImage(systemName: "chevron.backward")!
        let coloredImage = image.withTintColor(.label)
        let finalImage = coloredImage.withConfiguration(config)
        
        return finalImage
    }()

* Edit: I'm trying to set that image as UIBarButtonItem