Two aspects. First is finding the metadata, it is only fast the second time you look it up. The first time you tend to have to pay for a handful of page faults to get the data from the assembly into RAM. It is cached after that. You tend to care about (or measure) the first time.
Second one is that the directly calling a method or accessing a field or property is so incredibly fast. It doesn't typically cost more than one or two cpu cycles. Including none when the method can be inlined or the field access can be overlapped with another instruction. Reflection will always compare poorly against that, it takes hundreds of instructions.
Reflection is a suitable solution when other code takes a substantial amount of time so the cost of reflection is a small factor. Which includes anything that involves I/O like file formats and dbase mappings. And code that runs at human time, like designers and compilers.