Setting specific PHP version with SetHandler in .htacess per directory doesn't work

Question

Using Ubuntu 18 & Apache, I am trying to achieve state in which a PHP handler of my choice is used in each directory.

There is a .htaccess directive for that which itself works fine - I can change x-httpd-php to x-httpd-php-source and it is correctly applied

<FilesMatch \.php$>
    SetHandler application/x-httpd-php
</FilesMatch>

So I thought I would set a specific one similary as in tutorials for AddHandler

<FilesMatch \.php$>
    SetHandler application/x-httpd-php73
</FilesMatch>

Sadly, this does nothing (raw PHP code is displayed)

I have PHP7.1 - PHP7.3 installed and modules are enabled with sudo a2enmod php7.*

EDIT: I was able to do this with SetHandler "proxy:unix:/var/run/php/php7.3-fpm.sock|fcgi://localhost/" after some extra package installing , however I would like to know how to force PHP version during SetHandler in a way I described.