0

I have a scala function and want to create a consolidated span and child span for where ever this function is called in stackdriver (gcp) using cloud logging and opencencus.

What I have so far:

def merge(fileTables: FileTables)(implicit spark: SparkSession):
DataFrame = fileTables("test").withColumn(
"loan_status",
when('loan.isin(coerceStatuses: _*), lit("Paid Off")).otherwise('loan_status)
).withColumn(
"as",
lit(CLIENT_CONFIG.MAX_DATE.map(_.toString).orNull)
)
def myFunction(): Any = {
val tracer = Tracing.getTracer()
val exporter = StackdriverStatsExporter.createAndRegister()

//val tracer: Tracer = Tracing.getTracer()
val span = tracer.spanBuilder("testSpan").startSpan()
try {
/* call function here */
} finally {
span.end()
}
}

Need a child trace for whenever the merge function is called:

val loanDataTablePreStatics = merge(resolveUpstream(job))

Imports used:

I have a scala function and want to create a consolidated span and child span for where ever this function is called in stackdriver (gcp) using cloud logging and opencencus .

What I have so far:

def merge(fileTables: FileTables)(implicit spark: SparkSession):
DataFrame = fileTables("test").withColumn(
    "loan_status",
    when('loan_status.isin(coerceStatuses: _*), lit("Paid Off")).otherwise('loan_status)
  ).withColumn(
    "as",
    lit(CLIENT_CONFIG.MAX_DATE.map(_.toString).orNull)
  ).withColumn(
    "loan",
    when('loan_status.isin(coerceStatuses: _*), lit(0.0)).otherwise('loan_balance_current)
  ).recalcCarryThrough()

 def myFunction(): Any = {
    val tracer = Tracing.getTracer()
    val exporter = StackdriverStatsExporter.createAndRegister()

//    val tracer: Tracer = Tracing.getTracer()
    val span = tracer.spanBuilder("testSpan").startSpan()
    try {
      /* call function here */
    } finally {
      span.end()
    }
  }

Need a child trace for whenever the merge function is called val loanDataTablePreStatics = merge(resolveUpstream(job))

Imports used:

import com.google.cloud.MonitoredResource,
import io.opencensus.trace.{Span, Tracer, Tracing},
import com.google.logging.`type`.LogSeverity.INFO,
import com.google.cloud.logging.{LogEntry, Logging, LoggingOptions, Severity},
import com.google.cloud.logging.Payload.StringPayload
Mazlum Tosun
  • 5,761
  • 1
  • 9
  • 23

0 Answers0