-1

This question is a little complicated but I will do my best to make it simple.

I have a program which I want to run multithreaded.

This is what the program does:

  1. initializes an executable (commandline utility)
  2. loads a file into the executable (files are provided from a data provider method)
  3. sends commands to that executable based on the file which was loaded
  4. parses the received responses from executable
  5. writes results to a csv file

All this takes place in a single method.

However when running in multithreaded mode, everything runs fine except, all the results written to the csv file are wrong and out of order.

However when I added the keyword sychronized in the method declaration and run the program with multiple threads, the program works just fine. 

public sychronized void run(Dataprovider data) {
    ...
}

However the program runs at the same speed as if I were running in single thread mode. How can I fix this? This is driving me nuts...

How can I run this program properly multithreaded?

I'm looking for ideas and/or guidance

Edit:

However when running in multithreaded mode, everything runs fine except, all the results written to the csv file are wrong and out of order.

I load a file in the executable, I run some calculations on that file, then save it. I then get the file size in bytes (file.length) for that newly generated file. I compare the results of that new file with the old file (file which was loaded) and I see that the new file is smaller than the old file (which is totally wrong). The file sizes for the new file is consistently 12263 bytes, which is incorrect

Edit:

Here is a partial code which does the writing to CSV file:

Edit: Removing code example for simplicity

Moonie Asemani
  • 375
  • 1
  • 4
  • 13
  • 1
    Depends on what "all the results written to the csv file are wrong and out of order" means. – Joe C Nov 13 '16 at 18:12
  • how are you running this in multi-threaded mode? i would expect that when running in multi-threaded mode that all the results are out of order -- once multiple threads are running, they're not executing in any predictable order. – matias elgart Nov 13 '16 at 18:14
  • @JoeC Once I load a file in the executable, I run some calculations on that file, then save it. I then get the file size in bytes (file.length) for that newly generated file. I compare the results of that new file with the old file (file which was loaded) and I see that the new file is smaller than the old file (which is totally wrong). – Moonie Asemani Nov 13 '16 at 18:15
  • What does "wrong" mean? – Joe C Nov 13 '16 at 18:17
  • @melgart I have a java wrapper built around that executable which allows me to interface with the executable via java. In that wrapper, I have put in an "enginepool" concept, which allows me to build up a pool of this execubale instances and I get() an instance for every file I load. The machine I run this on has 8 cores, so I have 8 instances of this executable. So I can run 8 files simutaneously – Moonie Asemani Nov 13 '16 at 18:18
  • @JoeC the file size is written to the csv file is consistently 12263 bytes for a majority of the files (thousands) – Moonie Asemani Nov 13 '16 at 18:19
  • But you haven't told me what *inside* the file is wrong. – Joe C Nov 13 '16 at 18:22
  • @JoeC hmm..you think the content of the new file might have an issue? I will check – Moonie Asemani Nov 13 '16 at 18:24
  • Well, if the file size is different, does that not mean the contents are different? – Joe C Nov 13 '16 at 18:24
  • @JoeC I just checked multiple files and have verified the contents are correct. I also manually checked the file sizes of the generated files and those are different from what is reported on the csv file. I am thinking my problem may possibly be that I am writing multiple results (up to 8) simutaneously to a single file...perhaps some collision is hapening. My eclipse debugger does not accomodate multithreaded debugging, else I don't know how to use it – Moonie Asemani Nov 13 '16 at 18:33

1 Answers1

1

However when running in multithreaded mode, everything runs fine except, all the results written to the csv file are wrong and out of order.

I can make sore guesses as to what you mean by this statement, but it would help if it were more specific.

Is it the case that the results are wrong because outputs from different threads are being jumbled together into the same line or even the same token within a line?

In a csv file, the records are typically separated by newline characters. Can you refactor your solution so that a thread produces a complete line before writing to the output, and writes that line all in one go to the output?

Does your solution already do it that way? (It's not clear... there is no code in the question.)

Drew Wills
  • 8,408
  • 4
  • 29
  • 40
  • I have edited my question to be more clear. I would rather not add code because it would rather complicate things. The format of the csv file is correct. However the data written to it is not. I look for file size of a file I generate, the file size is consistently 12263 bytes for a majority of the files, which is not correct. – Moonie Asemani Nov 13 '16 at 18:27
  • That is an interesting suggestion, writing to the output all in one go...I may have to look into that. I think the problem I have is that I have multiple file results being written simultaneously to a single file, which may cause collison. – Moonie Asemani Nov 13 '16 at 18:29
  • Again -- I can't see the code; but I'm beginning to suspect you have some state-tracking variables that are being written (overwritten) by multiple threads. – Drew Wills Nov 13 '16 at 18:31