Best way to work on multi-line output from external process called from groovy script?

Question

I use this method:

def runProcess(List cmd) {
  def process = cmd.execute()
  def output = new StringWriter(), error = new StringWriter()
  process.waitForProcessOutput(output, error)
  def exitCode = process.exitValue()
  if (exitCode) {
    throw new Exception("Error: $error with code: $exitCode")
  }
  return output.toString().split()
}

To run external processes that can return one or multiple lines of output.

Sometimes I need to inspect each line and return a match if found. First I tried with an eachLine closure but found that I cannot return from that (Can you break from a Groovy "each" closure?):

def sample() {
  String tagName = ""
  def tags = runProcess(["git", "tag"])
  tags.eachLine { tag -> 
    println "tag $tag"
    if(tag = "mytag") {
      tagName = tag
      // Cannot return from a eachLine closure :-(
      return tag
    }
  }
  return tagName
}

The above will work but if I have 1000 lines it will go through all of them - since the return statement is ignored.

I am now trying with a classic for loop:

def sample() {

  def tags = runProcess(["git", "tag"])
  println "tags.getClass() "  + tags.getClass() // this is java.lang.String
  String[] tagsArr = tags.split("\n");
  println "tags.getClass() "  + tagsArr.getClass() // this is [Ljava.lang.String
     
  if (tagsArr != null && tagsArr.length > 0) { // does NOT = true when tagsArr is empty :-(
     for (String tag : tagsArr) {
        def desc = shellCommand(["git", "describe","--tags","$tag"])
        if(desc.trim() == "mytag") {
          println "Found tag: $tag at HEAD"
          return tag
        }
     }
  }
}

This is pretty verbose/ugly and does not work when tagsArr is empty (still investigating this and any input is appreciated!).

Any suggestions on how to implement better handling of multi-lines output from calling an external process?

Also this from above:

return output.toString().split()

does not seem right...

I have also looked at:

http://konstructcomputers.blogspot.com/2013/12/groovy-line-by-line-process-output.html

which looks quite verbose/extensive. I hoped that groovy offered some minimal way of doing this to avoid this kind of "low level" boilerplate code.

Sidenote. Interestingly enough this page:

http://docs.groovy-lang.org/latest/html/documentation/working-with-io.html

does not mention waitForProcessOutput which I thought was the most robust way to get output from an external process.

UPDATED Example (with readLines and find)

Below an update example based on suggestion below. I modified the original example to make it more clear.

Expected result = annotated

$ git for-each-ref refs/tags
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/001
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/002
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/003
706a21e04441c43e2b0372bd8607be74c0377690 tag    refs/tags/annotated
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/lightweight

Using find() and split() (working)

def runProcess(List cmd) {
  def process = cmd.execute()
  def outputWriter = new StringWriter(), errorWriter = new StringWriter()
  process.waitForProcessOutput(outputWriter, errorWriter)
  String output = outputWriter.toString()
  String error = errorWriter.toString()
  int exitCode = process.exitValue()
  if (exitCode) {
    throw new Exception("Error: $error exit code: $exitCode")
  }
  // Notice split()
  return output.split()
}

def sample() {
  def tags = runProcess(["git", "tag","--points-at","HEAD"])
  def result = tags.find{tag -> 
    def tagType = runProcess(["git", "cat-file","-t","$tag"])
    if(tagType[0] == "tag") {
      return tag
    }
  }
  return result
}

assert(result == "annotated")

readLines() and find() (also working)

def runProcess(List cmd) {
  def process = cmd.execute()
  def outputWriter = new StringWriter(), errorWriter = new StringWriter()
  process.waitForProcessOutput(outputWriter, errorWriter)
  String output = outputWriter.toString()
  String error = errorWriter.toString()
  int exitCode = process.exitValue()
  if (exitCode) {
    throw new Exception("Error: $error exit code: $exitCode")
  }
  // Notice NO split()
  return output
}

def sample() {
  def tags = runProcess(["git", "tag","--points-at","HEAD"])

  def result = tags.readLines().find{ tag ->
    def tagType = runProcess(["git", "cat-file","-t","$tag"])
    if(tagType.trim()  == "tag") {
      return tag
    }
  }
  return result
}

assert(result == "annotated")

Answer 1

String sample() {
  def tags = runProcess(["git", "tag"]) // expected multiline string to be returned
  tags.readLines().find{tag->
    def desc = ...
    return desc=="mytag" // will break on first true returned
  }
}

here is a test script you could run in any online console:

def tags = '''
aaa
bbb
ccc
'''

def found = tags.readLines().find{tag->
    return "BBB" ==  tag.toUpperCase()
}

println "found: ${found}"

https://groovyconsole.appspot.com/edit/5201583847505920?execute

another example based on updated question:

def output = '''
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/001
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/002
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/003
706a21e04441c43e2b0372bd8607be74c0377690 tag    refs/tags/annotated
67a27fec636a12346f391c67be01a0c8c0e1b7b8 commit refs/tags/lightweight
'''

def tagname = output
                .readLines()
                .findAll()             //keep only non-empty lines
                .collect{it.split()}   //split each line by spaces
                .findAll{it[1]=='tag'} //find all items with 'tag'
                .findResult{tag->
                    def result = true        //evaluate something here to find the one
                    if(result)return tag[2]  //search done - return the value
                    else return null         //continue searching
                }
assert tagname=='refs/tags/annotated'