Scalding write method not found in 0.15.0 version

Question

Just started to implement a simple scalding start-up program. Followed this documentation for references. In this first example it could not resolve write method as a syntax.

import com.twitter.scalding._
class WordCountJob(args: Args) extends Job(args) {
  TypedPipe.from(TextLine(args("input")))
    .flatMap { line => line.split("""\s+""") }
    .groupBy { word => word }
    .size
    .write(TypedTsv(args("output")))
}

build.sbt file and library dependencies

name := "SampleScaldingProject"

version := "1.0"

scalaVersion := "2.10.4"

resolvers ++= Seq( "maven.org" at "http://repo2.maven.org/maven2",
  "conjars.org" at "http://conjars.org/repo",
  "codahale.com" at "http://repo.codahale.com" )

libraryDependencies ++= Seq("org.scalatest" % "scalatest_2.10" % "2.1.0" % "test",
"com.twitter" % "scalding-core_2.10" % "0.15.0"
)

score 0 · Answer 1 · answered Dec 15 '15 at 13:31

Apparently that version changed, if you take a look at the WordCountJob.scala's version on github is different of your:

package com.twitter.scalding.examples

import com.twitter.scalding._

class WordCountJob(args: Args) extends Job(args) {
  TypedPipe.from(TextLine(args("input")))
    .flatMap { line => line.split("\\s+") }
    .map { word => (word, 1L) }
    .sumByKey
    // The compiler will enforce the type coming out of the sumByKey is the same as the type we have for our sink
    .write(TypedTsv[(String, Long)](args("output")))
}

And if you take a look at the API Scaladocs the stage .size return a UnsortedGrouped[K, Long] and this trait hasn't a write method. So I assume the API changed.

I hope this helps. Regards.

Scalding write method not found in 0.15.0 version

1 Answers1