Stack Overflow
Stack Overflow

Questions tagged [cdap]

CDAP exposes developer APIs (Application Programming Interfaces) for creating applications and accessing core CDAP services. CDAP defines and implements a diverse collection of services that support applications and data on existing Hadoop infrastructure such as HBase, HDFS, YARN, MapReduce, Hive, and Spark.

References

138 questions
1
vote
2 answers

CDAP Source plugin to read data from Sftp server

I want to read a csv file that is available to Sftp server by using a cdap source plugin. I came across FTP Batch Source plugin that does the same. But when running this i am getting below exception. Caused by: java.io.IOException: No FileSystem for…
Neelesh
  • 666
  • 6
  • 17
1
vote
1 answer

do the meta-data / data-lineage app have APIs for external datasources?

Our project is on AWS and we are already using different AWS services for our data-pipeline. So Would like to know if we can push those info into your meta-data/Data-lineage(FFL) apps? -VVS
Venkat.V.S
  • 349
  • 3
  • 7
1
vote
1 answer

How can I import CDAP Pipeline other than the UI?

Is there a way to import pipelines using CDAP CLI? I am interested in automated pipeline deployment.
pmatthew
  • 23
  • 7
1
vote
1 answer

Possible to modify or delete rows from a table in BigQuery dataset with a Cloud Data Fusion pipeline?

I have a requirement to build a Data Studio dashboard and to use data from BigQuery dataset. I have imported my data to BQ using Data Fusion from an on-premise MS SQL server, and the requirement is I have to delete the last 5 days of the records and…
Bluescrod
  • 81
  • 1
  • 7
1
vote
1 answer

Equivalent to LEFT() function in Google cloud data fusion data pipeline wrangler?

I need to port recipes from dataprep recipe into data fusion pipeline wrangler. derive type: single value: LEFT(column3, 3) as: 'column3' When I enter custom transform it gives me an error: io.cdap.wrangler.expression.EL.compile@1:6 unsolvable…
AntoinetteF
  • 21
  • 2
1
vote
1 answer

table-lookup directive can not find an existing table where preparing data

Hi: I am new to use CDAP(Cask™ Data Application Platform). I have parsed the states.json file and successfully wrote the CDAP Table Dataset named "StateTableDataset1". When I execute table-lookup state StateTableDataset1 in the customer tab, an…
bigtree
  • 43
  • 5
1
vote
1 answer

Parameter subscription must conform to pattern ^projects/[^/]+subscriptions/[^/]+$

.I am trying to create a data fusion pipeline which fetches data from pub sub and the used projection to convert message into string (as told by you) and thens ave it to GCS. howeer,its giving me error: Parameter subscription must conform to pattern…
user3746835
  • 11
  • 3
1
vote
1 answer

CDAP converting Dates in this format "/Date(1577746800000+0100)/" to simple date

Hello i would like to convert dates in this format /Date(1577746800000+0100)/ to simple date using CDAP. someone can help me? thanks example of…
nuxer
  • 17
  • 5
1
vote
1 answer

Does CDAP support multiple inputs in SparkCompute and SparkSink plugins?

I am looking for a way to implement a SparkCompute (or SparkSink) plugin that consumes from multiple inputs. Looking at the interface, both SparkCompute and SparkSink plugins are limited to consume only one. This is an excerpt from…
egordoe
  • 918
  • 1
  • 5
  • 12
1
vote
1 answer

Is there any way to call Bing-ads api through a pipeline and load the data into Bigquery through Google Data Fusion?

I'm creating a pipeline in Google Data Fusion that allows me to export my bing-ads data into Bigquery using my bing-ads developer token. I couldn't find any data sources that should be added to my pipeline in data fusion. Is fetching data from API…
krishr2d2
  • 19
  • 2
1
vote
2 answers

How do I preview data in a JSON file that resides in GCS?

I'm playing with Google Data Fusion for the first time (it got released into beta about 4 hours ago) and the first thing I'm trying to do is something fairly dumbed down - read the contents of a JSON file that resides in a GCS bucket. The GCS source…
jamiet
  • 10,501
  • 14
  • 80
  • 159
1
vote
1 answer

Cloudera and Cdap integration

How to add CDAP(cask data application platform) in cloudera? I installed cloudera in my machine and added a node to it. Now I want to add cdap to cloudera in my node using cloudera manager.
Gopirengaraj
  • 43
  • 6
1
vote
2 answers

unable to start cdap standalone server

I have cdap-sdk-3.0.0. When I try to start cdap server with instructions mentioned at location below: http://docs.cask.co/cdap/current/en/developers-manual/getting-started/start-stop-cdap.html I am getting below on console- G:\Bigdata…
S Singh
  • 1,403
  • 9
  • 31
  • 47
1
vote
2 answers

maven build failure- Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin

I am trying to build CDAP source code by following article from below location- https://github.com/caskdata/cdap But when I try mvn clean package, It will gives build error- . . kConfigurer.java:69: First sentence should end with a…
S Singh
  • 1,403
  • 9
  • 31
  • 47
0
votes
0 answers

CDAP execute crud queries

Can we read a each record in file at source and execute individual mysql crud (ex: update / delete) queries for each record at sink? Thanks & Regards, Ajay Babu Maguluri.
Ajay
  • 47
  • 4
1 2 3
…
9 10