Strongly typed IDs with Refit

Question

I am wondering how to get strongly typed IDs to work with Refit. I am using this package to automatically generate all needed things for the IDs. Here are the classes:

[StronglyTypedId(typeof(Guid))]
public partial struct UserId
{
    public partial class UserIdTypeConverter { }
    public partial class UserIdJsonConverter { }
    public partial class UserIdNewtonsoftJsonConverter { }
}

public interface IAudioSourceRefitClient
{
    [Get($"{Routes.AudioSourceRoutes.BaseRoute}/{Routes.AudioSourceRoutes.GetAllByUserRoute}")]
    Task<IEnumerable<AudioSource>> GetAllByUserId(UserId userId, CancellationToken cancellationToken);
}

A helper for the Refit settings:

public class RefitConfigurator
{
    public RefitSettings GetRefitSettings(IEnumerable<JsonConverter> converters, params Assembly[] assemblies)
    {
        var knownTypes = new List<Type>();
        knownTypes.AddRange(assemblies.SelectMany(assembly => assembly.GetTypes()));

        // Get all known converters from assemblies
        var foundConverters = assemblies.SelectMany(assembly => assembly.GetTypes())
            .Where(type => type.IsSubclassOf(typeof(JsonConverter)))
            .Select(type => (JsonConverter)Activator.CreateInstance(type)!)
            .ToList();

        return new()
        {
            ContentSerializer = new NewtonsoftJsonContentSerializer(new()
            {
                Converters = converters.Concat(foundConverters).ToList(),
                TypeNameHandling = TypeNameHandling.Objects,
                SerializationBinder = new KnownTypesBinder(knownTypes)
            }),
        };
    }
}

And the registration of the Refit client:

// Note: builer is from Autofac

builder.Register<IAudioSourceRefitClient>(context =>
        {
            var refitClient = RestService.For<IAudioSourceRefitClient>(new HttpClient()
                {
                    BaseAddress = new(libraryUrl)
                },
                refitSettings
            );

            return refitClient;
        });

This works quiet fine as long as I define the "userId" as a Guid. Preferably when working with strongly typed IDs I want to use the UserId here, but this results in Exceptions which I think Refit is producing.

Refit.ApiException: Response status code does not indicate success: 400 (Bad Request).
         at Refit.RequestBuilderImplementation.<>c__DisplayClass14_0`2.<<BuildCancellableTaskFuncForMethod>b__0>d.MoveNext() in /_/Refit/RequestBuilderImplementation.cs:line 288
      --- End of stack trace from previous location ---

I guess Refit is not able to convert the UserId correctly to a Guid, even that I have included the Converters within my RefitSettings. Do I miss something here or is Refit not meant to be used with this feature?