How to parse a string with filter criteria in scala and use it to filter objects

Question

Let's say I have an array of objects that contains some String, Integer and Enum values. And also contains arrays of these types and methods that return these types.

For example an array containing the following ExampleObject:

object WeekDay extends Enumeration { type WeekDay = Value; val Mon, Tue, Wed, Thu, Fri, Sat, Sun = Value  }

class ExampleObject (val integerValue1 : Integer, val integerValue2 : Integer, val stringValue1 : String, val weekDay: WeekDay.Value, val integerArray : Array[Integer])
{  def intReturningMethod1()= {0}  }

From the command line I pass in a string with filter criteria to the scala application. For example:

-filter_criteria "((integerValue1 > 100 || integerValue2 < 50) && (stringValue1 == "A" || weekDay != "Mon")) || (integerArray(15) == 1) "

The operators should do what you expect in a normal if statement with these types of values.

How can I parse the filter criteria string and use it to filter ExampleObjects from an array?

Or where should I start reading to find out how to do this?

Do you want to pass arbitrary scala statements or just simple filter criteria as in your example? — stefan.schwetschke, Feb 05 '14 at 16:00
With arbitrary statements, you have to use some kind of eval function (like in the answer from chrisloy). This gives you great flexibility needs a Scala compiler during runtime and opens a huge security hole. With simple criteria you can write your own little parser. Chossing one of the options makes a big difference in the solution. — stefan.schwetschke, Feb 05 '14 at 16:45
I would like to create a parser but I am not sure how to do this. Can you point me in the right direction? — WillamS, Feb 05 '14 at 17:03

score 3 · Accepted Answer · edited Sep 07 '18 at 15:16

If you want to restrict the input to a limited language, you can easily create a parser for that language using only the Scala core library.

I have done this for a stripped down version of your example

  object WeekDay extends Enumeration {
    type WeekDay = Value; val Mon, Tue, Wed, Thu, Fri, Sat, Sun = Value
  }

  case class ExampleObject(val integerValue1 : Integer, val stringValue1 : String, val weekDay: WeekDay.Value){
    def intReturningMethod1()= {0}
  }

First I use an import and create some helpers:

  type FilterCriterion = ExampleObject => Boolean
  type Extractor[T] = ExampleObject => T

  def compare[T <% Ordered[T]](v1 : T, c : String, v2 : T) : Boolean = c match {
    case "<" => v1 < v2
    case ">" => v1 > v2
    case "==" => v1 == v2
  }

  def compareAny(v1: Any, c : String, v2 : Any) : Boolean = (v1,v2) match {
    case (s1: String, s2:String) => compare(s1,c,s2)
    case (i1: Int, i2 : Int) => compare(i1,c,i2)
    case (w1 : WeekDay.WeekDay, w2 : WeekDay.WeekDay) => compare(w1.id, c, w2.id)
    case _ => throw new IllegalArgumentException(s"Cannot compare ${v1.getClass} with ${v2.getClass}")
  }

Then I create the parser:

  object FilterParser extends JavaTokenParsers {
    def intExtractor : Parser[Extractor[Int]] = wholeNumber ^^ {s => Function.const(s.toInt)_} |
      "intReturningMethod1()" ^^^ {(e : ExampleObject) => e.intReturningMethod1()}  |
      "integerValue1" ^^^ {_.integerValue1}
    def stringExtractor : Parser[Extractor[String]] = stringLiteral ^^ {s => Function.const(s.drop(1).dropRight(1))_} |
      "stringValue1" ^^^ {_.stringValue1}
    def weekDayExtrator : Parser[Extractor[WeekDay.WeekDay]] = stringLiteral ^? {
      case s if WeekDay.values.exists(_.toString == s) => Function.const(WeekDay.withName(s))_
    }
    def extractor : Parser[Extractor[Any]] = intExtractor | stringExtractor | weekDayExtrator

    def compareOp : Parser[FilterCriterion] = (extractor ~ ("<"| "==" | ">") ~ extractor) ^^ {
      case v1 ~ c ~ v2 => (e : ExampleObject) => compareAny(v1(e),c,v2(e))
    }

    def simpleExpression : Parser[FilterCriterion] = "(" ~> expression <~ ")" | compareOp
    def notExpression : Parser[FilterCriterion] = "!" ~> simpleExpression ^^ {(ex) => (e : ExampleObject) => !ex(e)} |
      simpleExpression
    def andExpression : Parser[FilterCriterion] = repsep(notExpression,"&&") ^^ {(exs) => (e : ExampleObject) => exs.foldLeft(true)((b,ex)=> b && ex(e))}
    def orExpression : Parser[FilterCriterion] = repsep(andExpression,"||") ^^ {(exs) => (e : ExampleObject) => exs.foldLeft(false)((b,ex)=> b || ex(e))}
    def expression : Parser[FilterCriterion] = orExpression

    def parseExpressionString(s : String) = parseAll(expression, s)
  }

This parser takes your input string and returns a function that maps an ExampleObject to a boolean value. This test function is constructed once while parsing the input string, using the pre-defined helper functions and the anonymous functions defined in the parser rules. The interpretation of the input string is only done once, while constructing the test function. When you execute the test function, you will run compiled Scale code. So it should run quite fast.

The test function is safe, because it does not allow the user to run arbitrary Scala code. It will just be constructed from the partial function provided in the parser and the pre-defined helpers.

  val parsedCriterion=FilterParser.parseExpressionString("""((integerValue1 > 100 || integerValue1 < 50) && (stringValue1 == "A"))""")

  List(ExampleObject(1,"A", WeekDay.Mon), ExampleObject(2,"B", WeekDay.Sat), ExampleObject(50,"A", WeekDay.Mon)).filter(parsedCriterion.get)

You can easily extend the parser yourself, when you want it to use more functions or more fields in your ExampleObject.

There were two minor bugs in my original answer. I fixed them in an edit. The code should now work without problems. Be careful to use the right version. — stefan.schwetschke, Feb 06 '14 at 16:01
I've combined the parser combinator with the rest of my program and it works really nice and fast! I would like to add an calculator to the intExtractor but I am stuck how to apply the operators to the ExampleObject => Double function http://stackoverflow.com/questions/21680403/scala-parser-combinator-based-calculator-that-can-also-take-a-datarecord — WillamS, Feb 10 '14 at 15:03

score 1 · Answer 2 · answered Feb 05 '14 at 16:04

1

You might want to have a look at Twitter's Eval utility library, which you can find here on GitHub. You could just substitute the passed in filtering logic into a String at the point you want to use it and pass it to the eval function.

answered Feb 05 '14 at 16:04

chrisloy

179
1
5

Isn't that kind of dangerous? I would rather limit the allowed code to values and methods from the object in combination with operators. – WillamS Feb 05 '14 at 16:11
Yeah it is pretty dangerous! Twitter designed this to be used to load in config; you mention passing arbitrary code in on the command line. I certainly wouldn't open it up to user input. – chrisloy Feb 05 '14 at 16:15
Unfortunately if you're not in a position to let the compiler do the work for you, I think you would have to write your own parser for that string, which would be a substantial amount of work, though admittedly much safer. – chrisloy Feb 05 '14 at 16:16

How to parse a string with filter criteria in scala and use it to filter objects

2 Answers2

Linked