There are two ways to do this. I'll create two separate answers to explain how.
The first technique is to create a custom Dockerfile (docs) which edits the www.conf (or any other file). You can also use the Dockerfile ADD command to add a complete file and override them.
In the case of this specific problem, we'll create a .ddev/web-build/Dockerfile with these contents:
# You can copy this Dockerfile.example to Dockerfile to add configuration
# or packages or anything else to your webimage
ARG BASE_IMAGE
FROM $BASE_IMAGE
ENV PHP_VERSION=7.4
RUN echo "security.limit_extensions = .php .html" >> /etc/php/${PHP_VERSION}/fpm/pool.d/www.conf
After you ddev start
you'll have the new configuration.
Instead of the RUN echo
approach that just appends to the file, given here for simplicity, you could RUN
a sed/awk/perl statement to change the file in place.
And alternatively you could put the version of the www.conf that you want into the .ddev/web-build directory and
COPY www.conf /etc/php/${PHP_VERSION}/fpm/pool.d/www.conf