Stack Overflow
Stack Overflow

Questions tagged [kylo]

Kylo is an open-source data lake management platform. Ask programming related questions here. For other topics refer to Google groups.

Kylo is an open-source data lake management platform.

Kylo offers a turn-key, enterprise-ready data lake solution that integrates best practices around metadata management, governance, and security gleaned from Think Big's 150+ big data implementation projects.

For non programming related topics refer to Kylo Community on Google groups.

68 questions
4
votes
1 answer

How to add a database source to Kylo

I’d like to create a new Data Ingest type feed in Kylo, to import data from an external database into Hive. On the second step of the feed wizard there’s a drop-down for Source Database Connection but my database isn’t listed. How do I add my…
harschware
  • 13,006
  • 17
  • 55
  • 87
3
votes
2 answers

What happened to the Teradata Kylo product?

GitHub activity is just empty last month, at least download links for the latest release 0.10.1 (March, 2019) lead to nowhere same thing for VirtualBox images - AWS S3 bucket does not exist My questions are: Is the project dead? Is there any…
Pavel Baravik
  • 173
  • 1
  • 9
3
votes
2 answers

Import Sqoop column names issue

I have a question on Kylo and Nifi. The version of Kylo used is 0.10.1 The version of Nifi used is 1.6.0 When we create a feed for database ingest (using database as source), in the Additional Options step there is no provision to enter the source…
Aparajitha
  • 31
  • 2
3
votes
2 answers

Split JSON into two individual JSON objects using Nifi

I have a JSON like { "campaign_key": 316, "client_key": 127, "cpn_mid_counter": "24", "cpn_name": "Bopal", "cpn_status": "Active", "clt_name": "Bopal Ventures", "clt_status": "Active" } Expected output 1st JSON : { …
Karthik Mannava
  • 207
  • 1
  • 5
  • 12
3
votes
1 answer

How do I transfer my cassandra data to pyspark using QueryCassandra and ExecutePySpark Nifi Processors?

I'm just querying cassandra table using querycassandra processor but what I'm not understanding is how do I pass my Json output file into ExecutePyspark processor as a Input file and later on I need to pass my Spark output data to Hive. Please help…
Karthik Mannava
  • 207
  • 1
  • 5
  • 12
3
votes
1 answer

Customize data-ingest template in kylo

I am trying to customize the standard data ingest template available in kylo using a configuration file. I added the field name nifi.getfile.business_line1=Test in application.properties file available in kylo-services/conf directory. When i try to…
Ravi Kiran Gururaja
  • 407
  • 7
  • 26
3
votes
1 answer

kylo ingestion feed error

I am new to Kylo. I manually deployed Kylo on a test cluster of Hortonworks HDP 2.5 and have reused my Nifi instance prior to kylo. I made a sample feed by following like ingestion tutorial (User Signups) and was successful. However, when I drop…
omer
  • 187
  • 6
  • 16
2
votes
1 answer

How to add a custom category property in Kylo?

I want to define some additional property while creating a category in Kylo. For example, a flag kind of thing. If the flag is Y, pass that category to some database as metadata for that category. Is it possible? If yes, please suggest how to do…
ashutosh tiwari
  • 65
  • 1
  • 6
2
votes
1 answer

Nifi processor is not parsing JSON correctly

I am using EvaluateJsonPath to extract one particular value from JSON. I am using the follwoing JSONPath expression: $.data[?(@.containerType == 'SOURCE' && @.path == 'SOURCE_KYLO_DATALAKE')].id This is the JSON document I'm calling the JSONPath…
Rakesh Prasad
  • 602
  • 1
  • 13
  • 32
2
votes
1 answer

Kylo security implementation in HDP KYLO sandbox?

I am trying to apply security in Kylo, such as the feed and category created by one user if it is not visible to other users. Is it possible in Kylo HDP Sandbox? If yes, what changes do I need to make?
ashutosh tiwari
  • 65
  • 1
  • 6
2
votes
1 answer

How do I read Kafka messages in consumer from last committed offset in Apache Nifi?

I have started my producer to send data to Kafka and also started my consumer to pull the same data.When I was using Consumekafka processor (kafka version 1.0) in Apache Nifi, I have few queries in my mind which are related to Kafka consumer. Q.1)…
Ashok Kuramdasu
  • 313
  • 4
  • 15
2
votes
1 answer

Merge flow files on condition based using nifi?

I had a 3 flowfiles which are coming from the same processor. FF1 -> {a:1,b:2,c:'name'} FF2 -> {a:1,b:5,c:'fruit'} FF3 -> {a:2,b:3,c:'abc'} By using MergeContent Processor I'm able to merge all the flow files, but my requirement is to merge flow…
Karthik Mannava
  • 207
  • 1
  • 5
  • 12
2
votes
1 answer

Kylo and nifi usage for ETL

We have started to explore and use Nifi for data flow as a basic ETL tool. Got to know about Kylo as a datalake specific tool which works over Nifi. Are there any industry usage and pattern where Kylo is being used Or any article giving its use…
tarunk
  • 549
  • 2
  • 7
  • 17
2
votes
1 answer

Kylo UI - Create new typescript module

I have a doubt about the way to work with Kylo and UI structure. According to the site http://kylo.readthedocs.io/en/v0.8.3/developer-guides/KyloDeveloperGuide.html?highlight=angular2 it says: Most of the Kylo UI depends on AngularJS and AngularJS…
Germanaz0
  • 914
  • 8
  • 18
2
votes
1 answer

NIFI: limit number of concurrent tasks of a NIFI processor in a NIFI-Cluster

The question says it all. How can I do one of the following things: How can I limit the number of concurrent tasks running for one processor cluster-wide? Is there any unique and short ID for the Node, I run on? I could use these ID to append to…
jboi
  • 11,324
  • 4
  • 36
  • 43
1
2 3 4 5