Stack Overflow
Stack Overflow

Questions tagged [scala-collections]

Collection library for Scala Programming Language

Related Tags:

1639 questions
0
votes
1 answer

Which elememt does DataFrame. DropDuplicate drop

If I sort a dataframe in descending ortder based on a column. And then drop the duplicates using df.dropDuplicate then which element will be removed? The element which was smaller based on sort?
Abhishek Tripathi
  • 1,211
  • 2
  • 15
  • 30
0
votes
1 answer

Spark 2.0: A named function inside mapGroups for sql.KeyValueGroupedDataset cause java.io.NotSerializableException

Anonymous function work fine. For following code set up the problem: import sparkSession.implicits._ val sparkSession = SparkSession.builder.appName("demo").getOrCreate() val sc = sparkSession.sparkContext case class DemoRow(keyId: Int, evenOddId:…
Y.G.
  • 661
  • 7
  • 7
0
votes
2 answers

comparing the index of element in scala list

I have 2 list val listA = List("Mary", "Harry", "Marry", "Harry", "Marry") val listB = List("Mary", "Harry", "Marry", "Harry", "Marry") Now I want to know whether the index of all occurrences of Harry in both the list are same or not. What is the…
Explorer
  • 1,491
  • 4
  • 26
  • 67
0
votes
1 answer

Retaining type information in List[Any]

I'm using certain external library that has a method which is overloaded several times with different arguments, something like: insertInto(index: Int, int: Int) insertInto(index: Int, lng: Long) insertInto(index: Int, dbl: Double) insertInto(index:…
JSelser
  • 3,510
  • 1
  • 19
  • 40
0
votes
1 answer

Iterate through complex Scala collections loaded from config file

I've read other threads on SO about iterating through collections from config files in Scala but they always assume the type in question is either a ConfigList or an ObjectList. In my case it is a more complex structure and I could not figure out…
iammyr
  • 271
  • 5
  • 13
0
votes
2 answers

How to create js.Array inside js.Array in scalaJS for highcharts as specified in the documentation?

I am working with highcharts in scalaJS. I want to create such [ [0, 1],[1, 2],[2, 8] ] JS array in scalaJS (basically 2D array) What kind of parameters can be passed can be seen in this documentation : HighChart Documentation Need to override…
Rupinder Singh
  • 233
  • 4
  • 9
0
votes
1 answer

Scala map co-occurence counts

I have a constantly-updating mutable.HashMap[String, String] with a record of current user locations: {user1 -> location1, user2 -> location4, user3 -> location4} I want to keep track of the location co-occurences between users - that is, how…
p3zo
  • 61
  • 1
  • 8
0
votes
2 answers

Scala RDD groupby count along with all columns

I need to get the all the columns along with the count.In Scala RDD. Col1 col2 col3 col4 us A Q1 10 us A Q3 10 us A Q2 20 us B Q4 10 us B Q5 20 uk A Q1 10 uk A Q3 10 uk A …
balaji
  • 11
  • 1
  • 3
0
votes
1 answer

Generate distinct values from a column in a spark dataframe

I have a spark dataframe like below id|name|age|sub 1 |ravi|21 |[M,J,J,K] I don't want to explode on the column "sub" as it will create another extra set of rows. I want generate unique values from the "sub" column and assign it to new column…
Puneeth Kumar
  • 171
  • 3
  • 15
0
votes
1 answer

Extract a column value and assign it to another column as an array in Spark dataframe

I have a Spark Dataframe with the below columns. C1 | C2 | C3| C4 1 | 2 | 3 | S1 2 | 3 | 3 | S2 4 | 5 | 3 | S2 I want to generate another column C5 by taking distinct values from column C4 like …
Puneeth Kumar
  • 171
  • 3
  • 15
0
votes
2 answers

Scala: How to mix up two List in proportion and in order

I want to define a function like this: def mixUp[A](lista: List[A], aprop: Int, listb: List[A], bprop: Int): List[A] lista and listb are two List that has the same generic. And the aprop,bprop meaning the proportion of lista and listb that…
Joe
  • 3,581
  • 4
  • 23
  • 28
0
votes
1 answer

How to compare two rdds using scala+spark.? There is no key in both the rdds?

I want to compare data in two RDDs. How can I iterate and compare field data in one RDD with field data in another RDD. below Eg files:` File1 f1 f2 f3 f4 f5 f6 f7 1 Nancyxyz 23456 12:30 NEWYORK 9000 xyz 2 ranboxys 12345…
Nathon
  • 165
  • 1
  • 4
  • 13
0
votes
1 answer

Invert MultiMap in functional way in Scala

In Scala, if we have a MultiMap which maps String to Set[String], for example: val a = Map( "Account1" -> Set("Cars", "Trucks"), "Account2" -> Set("Trucks", "Boats") ) What's an elegant way of inverting / reversing it to end up with: Map( …
David B
  • 455
  • 6
  • 13
0
votes
1 answer

Scala List - extract 'Date' and 'Adj close' values (columns)

I am trying to extract the 'Date' and 'Adj close' columns List(Date, Open, High, Low, Close, Volume, Adj Close 2016-11-10, 113.50, 115.440002, 111.50, 113.120003, 5700, 113.120003 2016-11-09, 113.00, 118.00, 113.00, 117.419998,…
199284kkfsa
  • 1
0
votes
3 answers

create a map from list in Scala

I need to create a HashMap of directory-to-file in scala while I list all files in the directory. How can I achieve this in scala? val directoryToFile = awsClient.listFiles(uploadPath).collect { case path if !path.endsWith("/") => { path…
Darth.Vader
  • 5,079
  • 7
  • 50
  • 90
1 2 3
…
99 100