It's worth bearing in mind the service limits for KeyVault:
Secrets, managed storage account keys, and vault transactions: 2,000
Maximum transactions allowed in 10 seconds, per vault per region
A subscription-wide limit for all transaction types is five times per key vault limit. For example, HSM-other transactions per subscription are limited to 5,000 transactions in 10 seconds per subscription.
The specifics can be found here: Azure Key Vault service limits
Depending upon your scalability targets you may also wish to consider Cosmos DB:
Cosmos DB isn't cheap, but it is very scalable and supports encryption at rest.
Encryption at rest is a phrase that commonly refers to the encryption
of data on nonvolatile storage devices, such as solid state drives
(SSDs) and hard disk drives (HDDs). Cosmos DB stores its primary
databases on SSDs. Its media attachments and backups are stored in
Azure Blob storage, which is generally backed up by HDDs. With the
release of encryption at rest for Cosmos DB, all your databases, media
attachments, and backups are encrypted. Your data is now encrypted in
transit (over the network) and at rest (nonvolatile storage), giving
you end-to-end encryption.
As a PaaS service, Cosmos DB is very easy to use. Because all user
data stored in Cosmos DB is encrypted at rest and in transport, you
don't have to take any action. Another way to put this is that
encryption at rest is "on" by default. There are no controls to turn
it off or on. Azure Cosmos DB uses AES-256 encryption on all regions
where the account is running. We provide this feature while we
continue to meet our availability and performance SLAs.
https://learn.microsoft.com/en-us/azure/cosmos-db/database-encryption-at-rest