Background
So I've been pulling my hair out the past few weeks trying to get OIDC authentication working based on ADFS in various applications, specifically Proxmox VE as well as Gitea. The reason why I am doing this is primarily driven by Proxmox, where I want to replace regular username/password authentication for our infrastructure admins with YubiKey + PIN. This variant is not natively supported by Proxmox, so I thought using ADFS as an OpenID provider would be a solution, as it supports smartcards and therefore YubiKeys.
What I've tried so far
Adding each application as "Server application accessing a web API" as well as simply "Server application". While the login with the latter one works, the returned username claims are gibberish, I also cannot control who is allowed to access the application. With "Server application + Web API" the basic authentication works, but the following userinfo request failes with:
MSIS9921: Received invalid UserInfo request. Audience 'microsoft:identityserver:<my-token>' in the access token is not same as the identifier of the UserInfo relying party trust 'urn:microsoft:userinfo'
Setting the "Access control policy" to "Permit everyone" does not make a difference.
Example configuration for Gitea, based on https://wiki.resolution.de/doc/openid-oauth-authentication/latest/setup-guides/adfs-setup-guide:
Server Application configuration - Web API - General (ID of server application added as relying party) - Web API - Client Permissions - Web API transform rules
Removing the server application identifier from the relying party identifiers allows login, but ignores any transform rules / access control polices, likely because ADFS cannot link the two apps together.
Other options
The only other option which would support authentication using YubiKeys only would be Azure AD, which we already use for various other applications successfully. The problem is, however, that the accounts with which we want to authenticate with (infra-admins) are not synced into Azure, to avoid a possible attacker also gaining foothold in our on-premises environment in case he takes over Azure. Unless I am overlooking something, AAD does not support authentication for unsynced users.
Am I doing something entirely wrong, or is this setup simply not working with applications unaware of ADFS? As I have searched around, one must provide some sort of "resource" parameter with the request "urn..." value when querying the userinfo endpoint, which I am not able to add.