Will using Scala in a more functional way (scalaz) incur a performance/maintainability penalty?

Question

I'm currently working on a small project (< 10k loc) which is mainly pure but relies on mutable optimizations mainly based on iterators and some data-structure reuse for heavy-duty calculations.

I'd like to learn a bit more functional programming and want to get more type safety, by e.g. wrapping mutable computations into state transformer monads and the like. For this purpose there exists the scalaz library.

Question One

When abstracting my computations on a larger scale by using all the fancy functional stuff, will I introduce performance killers that I won't get rid of? Like when my calculation is wrapped deep to the knees in Monads?

Question Two

Is it feasibly at all considering Scala's limited type inference? I'm currently fighting with very large type signatures (maybe because I don't know how to properly get rid of them). I suppose that going more "functional" will introduce even more such boiler-plate code.

Disclaimer

I'm not questioning whether the functional approach is good or bad. Asking this question for Haskell is pointless. I am questioning whether it is sensible doing so for Scala.

Edit on request: example of large type signatures in my project

(but this would be a different question)

The following code describes an iterative computation on a type-parameterized input object (DiscreteFactorGraph[VariableType, FactorType[VariableType]]). You can construct a computation object with createInitialState and perform computation on it with advanceState and finally extract some information from it with marginals.

I want the type of the factor graph object (and its parameter types) to be preserved during the computation so that the final application of marginals yields the correct type of DiscreteMarginals[VariableType]. I think currently I only have to preserve the variable type inside the computation type (which is TState), so carrying around the factor type is not used. But at a different place I need even the type of DiscreteFactorGraph to be variable, so I tend to need more type information carried through the computation in the future.

I was fiddlying around with this part a lot and I hope there's some better solution. Currently I have a pretty functional approach, where there are only those three functions. But I have to chain the type through them. Alternatively I can define it as a class and parameterise the class with all those types, so I don't have to repeat the type parameters for each method.

object FloodingBeliefPropagationStepper extends SteppingGraphInferer {
  def marginals[V <: DiscreteVariable, F <: DiscreteFactor[V]](state: FloodingBeliefPropagationStepper.TState[V,F]): DiscreteMarginals[V] =
    BeliefPropagation.marginals(state._1, state._2)

  def advanceState[V <: DiscreteVariable, F <: DiscreteFactor[V]](state: FloodingBeliefPropagationStepper.TState[V,F]): FloodingBeliefPropagationStepper.TState[V,F] = {
    val graph = state._1
    (graph,
      BeliefPropagation.computeFactorMessages(
      graph,
      BeliefPropagation.computeVariableMessages(graph, state._2, graph.variables),
      graph.factors))
  }

  def createInitialState[V <: DiscreteVariable, F <: DiscreteFactor[V]](graph: DiscreteFactorGraph[V, F],
                                                                        query: Set[V],
                                                                        random: Random): FloodingBeliefPropagationStepper.TState[V,F] = {
    (graph,
      BeliefPropagation.computeFactorMessages(
      graph,
      BeliefPropagation.createInitialVariableMessages(graph, random),
      graph.factors))
  }

  type TState[V <: DiscreteVariable, F <: DiscreteFactor[V]] = (DiscreteFactorGraph[V,F],Map[(F,V),DiscreteMessage])
}

Could you give one or few examples of your large type signatures? — CheatEx, Aug 17 '11 at 08:47
Your type signatures are only unmanageable because you've chosen incredibly long names like `FloodingBeliefPropagationStepper.TState`. And if you're after a high-performance solution, using `DiscreteVariable` instead of `Int` or somesuch is likely to cause the largest performance hit (unless you actually need to do very little with the variables). So far, the code looks like it's written to maximize abstraction not performance. So how worried should you really be about performance? — Rex Kerr, Aug 17 '11 at 15:55
@Rex `DiscreteVariable` is only a description of the variable itself and does not hold any state (the concrete data structures for holding the state of a computation are created at a lower level from the "problem specification" that is described by the graph). And indeed you're looking at an abstraction class and the computations are at a lower level, like in `BeliefPropagation`. — ziggystar, Aug 17 '11 at 16:43
@ziggystar - Okay, in that case it's rather hard to tell whether using Scalaz will negatively affect performance, because you haven't given any details about what sorts of computations are being performed. The answer is thus "maybe". — Rex Kerr, Aug 17 '11 at 17:34
@Rex Maybe there is an (almost) general answer to this question? Has scalaz been heavily used in some real production projects? Maybe someone who has done this can comment on his experience particularly adressing performance and succinctness of code. — ziggystar, Aug 18 '11 at 07:28
As someone who writes a great deal of high-performance code, I can assure you that if you are not carefully packaging your data into primitive types which you compute on mutably, your code will in most cases perform poorly compared to, say, C++. Scalaz is no exception. I cannot yet comment on the relative performance of Scalaz vs. non-Scalaz functional immutable styles, though, which is why this is a comment not an answer. — Rex Kerr, Aug 18 '11 at 14:56
In terms of maintainability, in my experience, doing things the "scalaz way" can mildly improve maintainability, but the philosophy that you *must* do things the "scalaz way" is a disaster for maintainability. — Owen, Jan 27 '12 at 05:39

score 5 · Answer 1 · answered Aug 17 '11 at 08:27

Regarding Question One:

There will be some overhead by wrapping your computations into monads, applicatives, functors and other functional vodoo. But so does wrapping your computations into procedures, methods, objects.

The question's core is, how small the computation has to be, so that the wrapping starts to be noticeable. That's something noone will be to tell you without knowing some details of your project. However due to Scala's hybrid nature you don't have to go monads all the way down. It's quite possible to use the scalaz-like style for the higher-level compositions of your computations and use locally-contained mutable state where performance requires it.

Regarding Question Two:

Since I have no clue of the nature of your type signatures, if scalaz will help you by generalizing computations or if you will have to type your way around Scala's limited support for partial type constructor application.

If your type signatures go out of hand, I suggest that you try to declare type alias in a package object and use these.

I was also thinking to apply the functional stuff only to the higher levels but wondered whether it is possible. Thanks for assuring me. — ziggystar, Aug 17 '11 at 08:34