1

I am tying to scan usb msc in stm32 for audio files. This mp3 files are scattered in many folders which are unknown to the application. First I scan for directories in the root folder and find folders then I scan folders for mp3 files. This is very time consuming and for depth of 8 folders with many files in each folder.

Is there any Way to Scan for just for folders which have mp3 file in them using a better approach. Directory structure for testing is something like this:

enter image description here

Nima Aghayan
  • 109
  • 1
  • 7
  • No - you need to go through the whole directory tree to find the files having a particular extension. – 0___________ Dec 26 '22 at 13:15
  • That is the problem I have a device which is running in a similar microcontroller and it scan 994 folders and which are very scattered and very complicated folder structure in just 4 seconds but my method of using 100 percent of cpu takes about 40 seconds. – Nima Aghayan Dec 26 '22 at 13:31
  • 2
    I do not know your method. Cant advise. Post your code – 0___________ Dec 26 '22 at 13:44
  • "better approach" than what? You have not told us your approach! You have a solution, it is not fast enough, and you are asking how to do it. There are not many ways, the chances are any solution will be the same as yours. We need to see your method to avoid reproduction and advise on optimising it rather then reinventing it. Any code accessing a file system will be limited by the performance of the file system, so there may not be much to do. Optimising the device i/o layer using DMA and/or interrupts is likely the solution. May not be faster, but it won't require 100% CPU. – Clifford Dec 27 '22 at 11:04
  • _" many files in each folder"_ - this may not be a code issue. The FAT filesystem will crawl to a near halt with more than about 100 files in a single folder. That is a FAT issue, nothing you can do except not put hundreds of files in a single directory. The lack of useful and specific quantitative information your question however just leaves us guessing where the bottle-neck is. – Clifford Dec 27 '22 at 18:11

1 Answers1

1

It is not clear what your problem might be since you have not provided any code to see how you are scanning, or quantative information on the file/folder structure ("many files" is rather vague), or even specified the media type used.

However a solution that might overcome all the variables of filesystem performance, hardware I/O, driver implementation and media type and make access more deterministic regardless, is to maintain a separate index file or database in a single file in the root directory to map each MP3 file to its path so you need only search the index/database for the MP3 you need (or use it to directly list all MP3's without scanning the file system).

If you maintain that file sorted (or a separate index file that is sorted) then you can use a binary search to find a specific file. Or simply use a real database - though that might be a rather heavyweight solution for this purpose. You might even load the metadata into memory for even faster access, and write it to the filesystem only if it changes.

Either way, the solution I suggest is to isolate your application from the variability of the filesystem/media, and the lack of scalability of FAT in general by maintaining your own "metadata" file(s) indicating what is stored and where so that you can use that to access the files directly without file system scanning using findfirst/findnext semantics, and recursion which is always best avoided, but is the obvious way to scan a directory tree.

Incidentally this is precisely how iTunes works for example. The "iTunes Library.xml" contains meta-data about "songs" including their location. Clearly you need not have anything quite that detailed, but the principle is the same and there may be merit in using XML or JSON for your application given a suitable library for updating and accessing such a file.

By doing that, the performance is more directly within your control rather than dependent on the filesystem, media and/or device driver level. However you still have some control/responsibility over the media and its interface (SPI, SDIO, USB or whatever), and the device I/O layer (DMA, interrupts, polled, bit-banged), and while you may have little control over the choice of FAT and the ELM FatFs implementation, you can certainly impact its performance greatly at the device driver, hardware interface and physical media level.

Clifford
  • 88,407
  • 13
  • 85
  • 165