I had same issue, and I found resolution without custom domain using following additional azure settings.
- create Azure WAF, add custom rules to deny if not in IP list - this is if you need ip whitelisting, useful if your main domain uses akamai or other edge routing to point to external hosting of subdomains, you can use whitelist to restrict access to the akamai or other servers, though this takes some big lists you must paste of ranges one row at a time. Set any other web app firewall rules you want enforced for allow/deny.
- Create Azure Front Door named like you want as an endpoint url e.g. myappfrontdoor will make myappfrontdoor.azurefd.net. in backend pool specify the your public-ip shared dns name (see step 3) like myapptest..cloudapp.azure.com.
- This is the important step : in Settings at top of front door designer, disable cert validation. in routing rules config, no condition, forward to backend pool setup in prior step. This ignores the fact that you cannot cert your cloudapp.azure.com endpoint, and wraps it with a *.azurefd.net certificate.
- In your azure firewall, Edit NAT rules, set rule name myapp-web-fd-... , tcp, ip address, 147.243.0.0/16 (this is Azure's front door backend ip range). destination should be the firewall's own public ip. destination port 443, translated address should be the target vm's azure internal ip, target port - service port.
Now you will have a site like myappfrontdoor.azurefd.net.
Note that Azure Front Door and WAF have their own pricing costs, so maybe it is cheaper for you to buy a domain. Hopefully you are also using Azure Firewall, though expensive. If not, one could point to public ip directly on NSG or on vm itself but I wouldn't skip having a firewall for a public server. There is a standing Azure enhancement request to get Azure Front Door to recognize certificates, but it was triaged 2 years ago and still not added, so not sure if it will be worked. If it ever does get worked, devs could make own cert auth and self-signed cert with expirations to more securely hook front door to azure internal vm. For now, have to rely on the front door backend setting, waf, and azure firewall to have these things routed.
There are some options in Akamai and other edge routing systems to import cert and self-created authority sort of, but I've not tried that yet, so cannot confirm this would cleanly wrap your azure site without cert errors. You can make a self-signed authority using openssl commands as noted in other posts out and about on the web.