1

I have two dates in Scala of format 'yyyy-MM-dd' and they both are the last day of a month (2015-05-31) and I want to find the month difference between then. I have the following code but it is not straighforward to find the month difference.

 val format = new java.text.SimpleDateFormat("yyyy-MM-dd")         
 val diff = format.parse(date1).getTime - format.parse(date2).getTime

 val days = TimeUnit.DAYS.convert(diff, TimeUnit.MILLISECONDS)

Any better suggestion?

HHH
  • 6,085
  • 20
  • 92
  • 164

3 Answers3

6

If you are using Java 8 or later, you can use the new Date and Time API in the java.time package.

First, you create two LocalDate objects:

import java.time._
val s1 = LocalDate.parse("2012-05-31")
val s2 = LocalDate.parse(otherDate)

Then you have two options. First option, use the ChronoUnit objects directly:

import java.time.temporal._
ChronoUnit.MONTHS.between(s1, s2)   // the answer you are looking for

Or create a Period object, which is used to represent periods of time.

val p = Period.between(s1, s2)
p.getMonths() // returns the number of months between the two dates

The Period class has other methods, such as getDays(), that enables you to obtain more information about the period between the two dates.

Basil Bourque
  • 303,325
  • 100
  • 852
  • 1,154
virsox
  • 463
  • 4
  • 12
  • 2
    Excellent suggestions. Please keep in mind that a `Period` acts as a remainder to larger units. `getMonths` on something that is 15 months apart will return 3 and `getYears` will return 2. Because of this, I prefer the `ChronoUnit.Months.between` solution which would return 15. – Shellum May 21 '19 at 23:23
2

There is a simple method there

import org.apache.spark.sql.functions
import spark.implicits._
val res=dataframe.withColumn("Month_Age",
functions.months_between(
  col("endDate"),
  col("startDate")
))

But before that you should have to parse your date string to date format if it is not in date format

You can check the schema by

dataframe.printschema()

root  |-- endDate: date (nullable = false)  

      |-- startDate_date: date (nullable = true)

      |-- Month_Age: Long (nullable = true)

You can use

from_unixtime(col("Date")/1000).cast("date")

if the date is in epoch format.

Comment for any doubts.Happy coding.

ADARSH K
  • 606
  • 1
  • 8
  • 21
1

There is a special method in JodaTime:

monthsBetween

Creates a Months representing the number of whole months between the two specified datetimes. This method corectly handles any daylight savings time changes that may occur during the interval.

codejitsu
  • 3,162
  • 2
  • 24
  • 38