7

Trying to figure out this problem for 2 weeks, but without any success. :X

It's occurring when I'm doing any kind of iteration, but mostly when using #forEach.

I'm not modifying the list, nor it's elements, in any way, so this seems very awkward to me. Example code:

    Map<Season, List<Team>> map = fetcher.getTeamsIn(ids);

    Set<Team> toCreateTeams = new HashSet<>();
    Set<Team> toUpdateTeams = new HashSet<>();

    map.forEach((k, v) -> {
        toCreateTeams.addAll(v.stream().filter(t -> !persistedTeams.containsKey(t.getId())).collect(Collectors.toSet()));
        toUpdateTeams.addAll(v.stream().filter(t -> {
            Date latestPersistedUpdate = persistedTeams.get(t.getId());
            return latestPersistedUpdate != null && t.getLastUpdated().after(latestPersistedUpdate);
        }).collect(Collectors.toSet()));
    });

map is instantiated in #getTeamsIn with new HashMap<>();

Tried to break on exception in eclipse to see if some thread was doing some crazy shit, but everything seems pretty normal to me. In the pics below, the exception was throw while iterating the map.

ConcurrentModificationException thread stack trace ConcurrentModificationException thread stack trace

I started having some other very strange behaviours too, like getting stuck in a lambda expression forever. In this case, it seems to me that Eclipse is stopping in the expression (for some unknown reason), as if some breakpoint was set in the line. When I suspend the execution and resume just the problematic Thread the flow just get backs to normal (until the next lambda expression) or some crazy ConcurrentModificationException.

stopping in expression for unknown reason

The whole thing seems like some Eclipse crazy bug to me, but I really don't want to rebuild my environment, if that's the case.

I'm using 

Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

on Linux Mint.

Thanks!

-- Update 1 --

Just to be clear: The error is occurring even for this simple example:

map.forEach((k, v) -> {
    System.out.println("Test" + k.getId());
});

Some random info that might be important: The exception only exploded after printing the last element of the map!

-- Update 2 --

Regarding the random info in Update 1, that's really unimportant, since, for performance reasons (at least in HashMap and ArrayList), ConcurrentModificationException is only checked in the end of iterations, by comparing the actual size of the array of elements with the expected size of it.

The code of the method #getTeamsIn:

public Map<Season, List<Team>> getTeamsIn(List<Season> seasons) throws InterruptedException {
    final CountDownLatch latch = new CountDownLatch(seasons.size());
    Map<Season, List<Team>> teamsInSeason = new HashMap<>();
    for (Season s : seasons) {
        httpclient.execute(new HttpGet(String.format(URL, s.getId())),
                new Callback(latch) {

                    @Override
                    public void completed(final HttpResponse response) {
                        super.completed(response);
                        try {
                            teamsInSeason.put(s, new TeamsUnmarshaller().unmarshal(response.getEntity().getContent()));
                        }
                        catch (IllegalStateException | IOException e) {
                            // TODO Auto-generated catch block
                            System.out.println(e);
                        }
                    }

                });
    }
    latch.await();
    return teamsInSeason;
}

Callback class just implements FutureCallback<HttpResponse> and countDown() the latch in all callback methods (#cancelled, #completed and #failed).

Kai
  • 38,985
  • 14
  • 88
  • 103
delta
  • 185
  • 2
  • 13
  • Could you include the declaration of `map`? – sprinter Apr 28 '15 at 02:55
  • Could you try `map.keySet().stream().forEach()` and `map.entrySet().stream().forEach()` to see if either set causes the same issue? – sprinter Apr 28 '15 at 03:00
  • is fetcher running in the background getting stuff while the other stuff is printing? – LhasaDad Apr 28 '15 at 03:24
  • @lhasadad, I use a `CountDownLatch` in #getTeamsIn to wait Apache httpasyncclient finish the requests. But fetcher is not running in background. – delta Apr 28 '15 at 03:29
  • Could you please try the standard iteration and not using a stream? – MJSG Apr 28 '15 at 03:43
  • @sprinter and @MJSG, yes for both forms. The problem is not restricted to HashMap, I tested with ArrayList (got exception at `java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1380)`) and For-Each loop too. I don't usually get the problem with small lists (+/- 500), don't know why. Something that the #getTeamsIn method is doing is causing the problem. I'll update the post with this method code soon. – delta Apr 28 '15 at 03:48
  • 1
    I don't see where you are calling countDown. Is it possible you are somehow calling countdown multiple times for some requests leading to the getTeamsIn method returning early and future request completions adding items to the map? – Pace Apr 28 '15 at 04:10
  • @Pace, that's make sense. Like I said in the Update 2, the `countDown()` is being called in the `Callback` class. It's just a simple class like that: `public class Callback implements FutureCallback { private final CountDownLatch latch; public OptaResponseCallback(final CountDownLatch latch) { this.latch = latch; } public void cancelled() { this.latch.countDown(); } public void completed(final HttpResponse response) { this.latch.countDown(); } public void failed(final Exception e) { this.latch.countDown(); } }` – delta Apr 28 '15 at 04:17

1 Answers1

3

Ok, just found out the problem. My overrided #completed method in the method #getTeamsIn is taking too long (thanks to JAXB) to return. Since the countDown() (being called in super.completed(response)) is before the teamsInSeason.put(s, new TeamsUnmarshaller().unmarshal(response.getEntity().getContent()));, we have a problem.

The fix is simple and ugly:

@Override
public void completed(final HttpResponse response) {
    try {
        teamsInSeason.put(s, new TeamsUnmarshaller().unmarshal(response.getEntity().getContent()));
    }
    catch (IllegalStateException | IOException e) {
        // TODO Auto-generated catch block
        System.out.println(e);
    } finally {
        super.completed(response);
    }
}

The weird Eclipse behaviour (getting stuck in some imaginary breakpoint for some unknown reason), if it persists, is a problem for another topic, I think.

Thank you all for the help and time!

delta
  • 185
  • 2
  • 13