I do not know the best way to compare the strings but a straightforward approach can look the following way
#include <stdio.h>
#include <ctype.h>
int main( void )
{
char filename[256] = "one.txt";
char shortname[11] = "ONE TXT";
char *p = filename;
char *q = shortname;
size_t i = 0;
while ( i < 8 && ( char )toupper( ( unsigned char )*p ) == *q ) ++i, ++p, ++q;
if ( *q == ' ' || i == 8 ) puts( "They coincide" );
}
The program output is
They coincide
That is the program checks whether initial characters of filename coincide with shortname. There can be more characters before the point in filname than there are characters in shortname.
Otherwise you can add condition *p == '.'