The reference is intended for application configuration, which is typically fetched and cached on startup or first use (the latter in this case). It's not intended to fetch every time. If you want lifetime management via ETags, for example, you might consider Azure Application Configuration. If you need those values to be encrypted at rest with only a limited set of people able to access, you'll not be able to use secrets this way but instead will have to call into Azure Key Vault yourself, but you should still cache the value for a limited time: Key Vault has very low rate limits, so if your function gets a lot of calls in a short period of time some of those may get throttled and fail (there is a built-in retry policy, but that won't help if requests keep getting queued up in rapid succession).
For example, you could just fetch the secret yourself (assumes C# since it wasn't evident what language you were using):
var vaultUri = Environment.GetEnvironmentVariable("AZURE_KEYVAULT_URL") ?? throw new InvalidOperationException();
var client = new SecretClient(new Uri(vaultUri), new DefaultAzureCredential());
KeyVaultSecret secret = await client.GetSecretAsync("my-secret");
There are a number of in-memory caching mechanisms you could use depending on how your function is triggered, such as a timer that fires every so often and updates the secret itself. Key Vault doesn't support ETags, so each request will be for the key regardless of whether it's changed. You could query if the version changed, but that saves you little (you'd have to fetch the full secret anyway, so that's 2 requests where you could just fetch the secret in 1 request at each interval).
