It is probably more complex than that and the best solution is as @sbass has advised, even if you do choose to change the file-system. You need to determine exactly where and why the latency is occurring before you blame uALFAT.
However for the record, I have successfully used ELM FatFs, and its cut-down ELM Petit FatFs, and also EFSL.
With respect to latency, while for example with ELM I have achieved write speeds of up-to 300 kbit per second using SPI (the speed is largely dependent on the card, and speeds can range from 30 kbit/s to 1 Mbit/s), I could not successfully use it to log a data stream of 96 kbit/s for any sustained length of time, even with optimised buffer sizes (multiples of a sector size), and a 50 kbit queue of 512 byte sectors. This was not down to the library, but rather the nature of the SD card, which apparently on 1 Mbit boundaries, would stall for up to 128 milliseconds, which was enough to exhaust the buffering provided by the queue. The solution is of course to increase the buffering as @sbas has said, but in this case the total system RAM was only 64 kbit so that was not possible.
If you can throw memory and an RTOS task (probably TI's own SYS/BIOS in your case) at the problem you should be able to get it working with the library you have. The amount of RAM necessary depends on the data rate and whether it bursts or is continuous.