How to apply multithreading in java to find a word in a directory(main directory and the word given) and recursively its subdirectories

Question

I am quite new on Stack Overflow and a beginner in Java so please forgive me if I have asked this question in an improper way.

PROBLEM

I have an assignment which tells me to make use of multi-threading to search files for a given word, which might be present in any file of type .txt and .html, on any-level in the given directory (So basically the entire directory). The absolute file path of the file has to be displayed on the console if the file contains the given word.

WHAT HAVE I TRIED

So I thought of dividing the task into 2 sections, Searching and Multithreading respectively, I was able to get the Searching part( File_search.java ). This file has given satisfactory results by searching through the directory and finding all the files in it for the given word.

File_search.java

public class File_search{
String fin_output = "";
public String searchInTextFiles(File dir,String search_word) {
    File[] a = dir.listFiles();
    for(File f : a){
        if(f.isDirectory()) {
            searchInTextFiles(f,search_word);
        }
        else if(f.getName().endsWith(".txt") || f.getName().endsWith(".html") || f.getName().endsWith(".htm") ) {
            try {
                searchInFile(f,search_word);
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            }
        }
    }
    return fin_output;
}

public void searchInFile(File f,String search_word) throws FileNotFoundException {
    final Scanner sc = new Scanner(f);
    while(sc.hasNextLine()) {
        final String lineFromFile = sc.nextLine();
        if(lineFromFile.contains(search_word)) {
            fin_output += "FILE : "+f.getAbsolutePath().toString()+"\n";
        }
    }
}

Now, I want to be able to use multiple threads to execute the task File_search.java using ThreadPoolExecuter service. I'm not sure If I can do it using Runnable ,Callable or by using a Thread class or by any other method? Can you please help me with the code to do the multi-threading part? Thanks :)

Adding multiple threads will not help and will make your code more complicated. The limiting factor in filesystem access is always I/O. — chrylis -cautiouslyoptimistic-, Sep 09 '21 at 04:53
Thank you for the reply chrylis-cautiouslyoptimistic. My assigment requires me to use ThreadPoolExecutor to search files. — NamitTheCoder, Sep 09 '21 at 06:05

score 1 · Accepted Answer · answered Sep 09 '21 at 06:53

I agree to the comment of @chrylis -cautiouslyoptimistic, but for the purpose of understanding below will help you.

One simpler approach could be to do the traversal of directories in the main Thread, I mean the logic which you have added in function searchInTextFiles and do the searching logic as you did in function searchInFile in a Threadpool of size let's say 10.

Below sample code will help you to understand it better.

public class Traverser {

private List<Future<String>> futureList = new ArrayList<Future<String>>();

private ExecutorService executorService;

public Traverser() {
    executorService = Executors.newFixedThreadPool(10);
}

public static void main(String[] args) throws InterruptedException, ExecutionException {
    System.out.println("Started");
    long start = System.currentTimeMillis();
    Traverser traverser = new Traverser();
    traverser.searchInTextFiles(new File("Some Directory Path"), "Some Text");
    
    for (Future<String> future : traverser.futureList) {
        System.out.println(future.get());
    }
    traverser.executorService.shutdown();
    while(!traverser.executorService.isTerminated()) {
        System.out.println("Not terminated yet, sleeping");
        Thread.sleep(1000);
    }
    long end = System.currentTimeMillis();
    System.out.println("Time taken :" + (end - start));
    
}

public void searchInTextFiles(File dir,String searchWord) {
    File[] filesList = dir.listFiles();
    for(File file : filesList){
        if(file.isDirectory()) {
            searchInTextFiles(file,searchWord);
        }
        else if(file.getName().endsWith(".txt") || file.getName().endsWith(".html") || file.getName().endsWith(".htm") ) {
            try {
                futureList.add(executorService.submit(new SearcherTask(file,searchWord)));
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }
}}

public class SearcherTask implements Callable<String> {

private File inputFile;
private String searchWord;

public SearcherTask(File inputFile, String searchWord) {
    this.inputFile = inputFile;
    this.searchWord = searchWord;
}


@Override
public String call() throws Exception {
    StringBuilder result = new StringBuilder();
    Scanner sc = null;
    try {
        sc = new Scanner(inputFile);
        while (sc.hasNextLine()) {
            final String lineFromFile = sc.nextLine();
            if (lineFromFile.contains(searchWord)) {
                result.append("FILE : " + inputFile.getAbsolutePath().toString() + "\n");
            }
        }
    } catch (Exception e) {
        //log error
        throw e;
    } finally {
        sc.close();
    }
    return result.toString();
}}

How to apply multithreading in java to find a word in a directory(main directory and the word given) and recursively its subdirectories

1 Answers1