0

I have program that uses 4 input .txt files. When program is started, certain data manipulation is done, data is grabbed from those 4 files, and program creates 2 output .txt files at the end. Program execution time at that time is about 18 seconds. I would like to know what would be fastest way to write this programs in terms of program execution? This is how program is written now:

  1. First, I started to read and write data sequentially, so my program logic is based on 2 parts - One loop (for reading necessary data and file creation) is used for first output file, and another loop (also for reading necessary data and file creation for that fole) is used for second output file. Second method needed to wait to first method to be done, with this approach. My total time of execution was at this point about 18 seconds.

  2. Then, I involved theads - I used 2 threadsm (one thread running it's loop) so every loop would be running in separate thread. With that approach, I cut total time of execution to about 9 seconds.

  3. Now; I started to ask If I could be speeding up total time of execution even more?

My code for the first thread looks like this (and the code for other thread is basically more or less similar):

Thread thread1 = new Thread() {
    public void run() {

        final List<Articles> article_list = ac.getFileValues("resources/articles.txt");

        String file_contents = "";
        String file1_data = "";
        for (int i = 0; i < article_list.size(); i++) {

            double price_art_local_val = cc.getPrice("resources/pricelist.txt", article_list.get(i).sifra);
            double[] art_all_shops = sc.getAmountInAllStores("resources/supply.txt", article_list.get(i).sifra);

            double total_value_art_all_shops_local = price_art_local_val * art_all_shops[0];
            double total_value_art_all_shops_foregin = total_value_art_all_shops_local / exchange_rate;

            file_contents = article_list.get(i).sifra+"\t"+article_list.get(i).naziv+"\t"+df.format(price_art_local_val)+"\t"+df.format(art_all_shops[0])+"\t"+article_list.get(i).jedinica_mjere+"\t"+df.format(total_value_art_all_shops_local)+"\t"+df.format(total_value_art_all_shops_foregin)+"\t"+df.format(art_all_shops[1])+"\n";

            System.out.print(file_contents);
            file1_data += file_contents;
        }

        if(file1_data != "")
        {
            save.saveFile("results/supply_value_articles.txt", file1_data);
        }
    }
};

The place I see further execution time reduction in my opinion is on methods cc.getPrice and sc.getAmountInAllStores() in that piece of code. My view is that It would be good to achieve that they run in separate threads, so other method would not have to wait execution of first method. Am I on good track?

So I presume that if I want to speed up execution in for loop I would need to make execution of methods cc.getPrice() and sc.getAmountInAllStores() also in separate threads. If that is not right solution, I would like to know what to do.

Then, if this is right solution, how can I achieve that? I do not know how to properly write code to use another thread, if that code is already in run() method. Can it be done even, I am not sure? Also, that methods return certain values, and they need to be stored in variables. So I would need also to get data for that metods stored; meaning threads would return data for me. I do not know how to done that properly. Seems like this won't help me then to basically write this (if I create one thread for method cc.getPrice, and another thread for method sc.getAmountInAllStores and name them thread3 and thread4):

thread3.join();
thread4.join();

I would need piece of code showing the appropriate solution. If I am not on the right track, and this can't be done (starting and using new threads inside running thread), please instruct me what to do. I have read some stackowerflow questions about BlockingQuene, but I think my approach brings something else that I need.

Please help with examples of code If you can. Thank you very much, help appreciated.

DarioBB
  • 663
  • 2
  • 8
  • 29
  • Measure the program by using a profiler, do not overthink where the bottlenecks may be. Also, please show proper code, like the declaration of `save` and the contents of `save#saveFile`. – Luiggi Mendoza Oct 25 '15 at 14:08
  • 1
    What do these two methods do? Their arguments seem to imply that they read from a text file. This means that for every article in the list, you read from two files, again and again and again. If that's true, that's what you must change. Read these two files in memory once and for all (into a HashMap with the sifras as keys, presumably), before the loop. I suspect doing that might remove the need for threads completely. – JB Nizet Oct 25 '15 at 14:10
  • 1
    Also, don't use += to build a long string in a loop. That's a performance disaster. Use a StringBuilder. – JB Nizet Oct 25 '15 at 14:13
  • Luiggi, I can't show all the code, but I can tell what it does. Save is just plain save of file using new BufferedWriter(new OutputStreamWriter(new FileOutputStream(filename), "UTF-8")); p.s. Luigi what is profiler? – DarioBB Oct 25 '15 at 14:13
  • JB Nizet, these to methods can't be outside for loop because I give them arguments from for loop - they don't execute same thing. They basically grab line (column) in .txt file which article information is stored. Parameter tells them about which article informations is needed, and that info occurs only once - I do not to twice for same article in for loop. Other method returns array. But I do not see how this could be done outside for loop because I need to return that specific column from line. – DarioBB Oct 25 '15 at 14:15
  • 2
    So, that's exactly what I presumed. Read every line of those two files and store them in a Map. Then, inside the loop, for each article, get the information from the map. Assuming there are 10,000 article in your list, you'll read each file 10,000 times instead of reading each once. That's where all the time is lost. – JB Nizet Oct 25 '15 at 14:19
  • JB Nizet, so you mean to store that data in map using for loop only once, something like this, before my for loop? for (int i = 0; i < file_1.size(); i++) { newmap.put(1, line_1_data); // line 1 } So I can later examine newmap variable and search for article code if exists in that map and then I get line instead I was doing? I that is what you ment, ok, I must say, I didn't saw that at start. Nevertheless, If I speed process with It, how to speed it even more? I want to know I there is possible using of threads inside run method which I described? What would be yet more flexible solution? – DarioBB Oct 25 '15 at 15:03

0 Answers0