Hudson SCM Polling Thread hungs while polling

Question

We use Hudson for our continuose build environment. For some reason, the thread for SCM Polling hungs somethimes after a while. I've experiemented a lot with the settings, but nothing seems to really work. How to fix this and are there some scripts out there which can detect such a case to be able to restart hudson? Btw. restarting hudson is the only way to solve this issue for us at the moment.

What SCM tool(s) are you using (svn, hg, etc) ? – pwan Apr 26 '11 at 04:17 — pwan, Apr 26 '11 at 04:17
Which hudson version? Can you also give us stack traces? – Robert Munteanu Apr 26 '11 at 07:57 — Robert Munteanu, Apr 26 '11 at 07:57

score 4 · Answer 1 · answered Apr 26 '11 at 07:56

That is similar to bug 5413, which should be solved since late 2010 with HUDSON 5977 (Hudson 1.380+, or now Jenkins).

You had in those thread some way to kill any thread stuck on the polling step:

very primitive (I'm too lazy to develop something better as this is not very important issue) Groovy script is bellow.
It could happened that it will kill also SCM polling which are not stuck, but we run this script automatically only once a day so it doesn't cause any troubles for us.
You can improve it e.g. by saving ids and names of SCM polling threads, check once again after some time and kill only threads which ids are on the list from previous check.

Thread.getAllStackTraces().keySet().each(){ item ->
  if( item.getName().contains("SCM polling") && 
      item.getName().contains("waiting for hudson.remoting")){ 
     println "Interrupting thread " + item.getId() item.interrupt() 
  }
}

score 0 · Answer 2 · answered May 22 '13 at 18:47

The other answer didn't work for me, but the following script found the the issue for this problem did:

Jenkins.instance.getTrigger("SCMTrigger").getRunners().each()
{
  item ->
  println(item.getTarget().name)
  println(item.getDuration())
  println(item.getStartTime())
  long millis = Calendar.instance.time.time - item.getStartTime()

  if(millis > (1000 * 60 * 3)) // 1000 millis in a second * 60 seconds in a minute * 3 minutes
  {
    Thread.getAllStackTraces().keySet().each()
    { 
      tItem ->
      if (tItem.getName().contains("SCM polling") && tItem.getName().contains(item.getTarget().name))
      { 
        println "Interrupting thread " + tItem.getName(); 
        tItem.interrupt()
      }
    }
  }
}

Hudson SCM Polling Thread hungs while polling

2 Answers2