Questions tagged [apache-atlas]

Apache Atlas is a data governance and metadata framework for Hadoop. Use for questions about setting up Atlas, the REST APIs, bridges, or problems encountered using Atlas.

Data Governance and Metadata framework for Hadoop

Features

  • Data Classification

Import or define taxonomy business-oriented annotations for data Define, annotate, and automate capture of relationships between data sets and underlying elements including source, target, and derivation processes Export metadata to third-party systems

  • Centralized Auditing

Capture security access information for every application, process, and interaction with data Capture the operational information for execution, steps, and activities

  • Search & Lineage (Browse)

Pre-defined navigation paths to explore the data classification and audit information Text-based search features locates relevant data and audit event across Data Lake quickly and accurately Browse visualization of data set lineage allowing users to drill-down into operational, security, and provenance related information

  • Security & Policy Engine

Rationalize compliance policy at runtime based on data classification schemes, attributes and roles. Advanced definition of policies for preventing data derivation based on classification (i.e. re-identification) – Prohibitions Column and Row level masking based on cell values and attibutes.

References:

107 questions
1
vote
2 answers

Can't execute command to set environment variables in docker container

I have seen the following links to execute multiple commands in docker-compose file: Docker-Compose + Command Using Docker-Compose, how to execute multiple commands docker-compose run multiple commands for a service which tell us how to execute…
Mostafa Ghadimi
  • 5,883
  • 8
  • 64
  • 102
1
vote
2 answers

Apache Atlas and Airflow Integration

I am trying to integrate an Apache Atlas instance I have running with Apache Airflow. Once I set up the connection in airflow.cfg I tried running a DAG from the Airflow scheduler. I get the following error in the log. [2021-02-02 20:50:47,958]…
r123
  • 33
  • 1
  • 5
1
vote
0 answers

Apache Atlas server not running,for check the reason 'zookeeper always send error message'

I am new for Atlas and had try to install it for 3 days.It's to hard for to install. So can anyone help me solve this question,thanks all. WARN - 2021-02-02 07:49:29.543; [ ] org.apache.zookeeper.ClientCnxn$SendThread; Session…
1
vote
0 answers

Apache Atlas and AWS S3

i am working on a project that has a requirement to store scientific data on AWS S3 as raw data for the beginning of a data lake. we are planning JSON for application data and using S3 metadata to persist application metadata (JSON schema) and…
rocketboy2000
  • 119
  • 1
  • 11
1
vote
0 answers

ReportLineagetoAtlas in Nifi not maintaing state of all nodes in Nifi Cluster

We have a NiFi cluster of 3 nodes having 100 flows deployed. We have configured a task ReportLineageToAtlas in NiFi which is syncing lineage information of flows to Apache Atlas. From the past few days it has stopped publishing lineage to Atlas. Few…
Aviral Kumar
  • 814
  • 1
  • 15
  • 40
1
vote
0 answers

apache atlas,When I used Impala, Spark and Presto to operate Hive, I found that I could not participate in the Hive hook competition

I'm testing the Apache Atlas Hive function. Our company operates hive with Presto Impala, and Atlas can't find the tables or blood ties created by Presto Impala. The reason is that Presto Impala will not produce Hive hook when operating Hive. Have…
fei wang
  • 11
  • 1
1
vote
0 answers

Apache Atlas with Cassandra and Elasticsearch

I am trying to run Apache atlas with cassandra as the graph storage backend and ElasticSearch as the index backend. I am getting a 503 error when hitting localhost:21000 from my browser. I don't see any errors in the application log or the start up…
Michael Scott
  • 540
  • 2
  • 8
1
vote
1 answer

Is it possible to integrate apache ranger/atlas with apache flink?

Flink is great at what it does and the flink sql client looks very promising. I'm trying to understand if it would be possible to integrate governance tools with flink sql client when used for interactive queries.
vikash dat
  • 1,494
  • 2
  • 19
  • 37
1
vote
0 answers

Update Classification using Apache ATLAS REST API

I would like to update tags associated to an entity using apache ATAS REST API. For example : Entity with this guid:81a38d33-a646-4bf6-9310-452947cc5a5a has three tags associated to it [main_tag,support_tag,conf_tag] I want to have new tags…
1
vote
1 answer

Apache atlas column lineage not found

I use Hive2.1.1 and Atlas2.0.0. I can found table lineage but not found column level lineage the detailed information is shown below For debug, I try Restful first, nothing about column lineage. [root@cent1 bin]# curl -X GET -u admin:admin…
Zeyu
  • 55
  • 10
1
vote
3 answers

Connection refused for apache atlas server

I have setup apache atlas on EC2. After setup , I started the apache atlas using command python2 atlas_start.py and it successfully started with below log: configured for local hbase. hbase started. configured for local solr. solr started. setting…
us56
  • 283
  • 1
  • 3
  • 12
1
vote
1 answer

can i restore soft deleted entity in apache atlas?

i want restore a soft deleted entity in apache atlas, cause i want to delete the classification tagged on that entity. i'm using atlas for data governance in HDP cluster. And i deleted an entity without deleting the classification tagged on that.…
PerleZhou
  • 11
  • 2
1
vote
1 answer

Ranger tag permissions appear to not take effect in Atlas

Having problem where it appears that policy tags set in Ranger appear to not take effect in Atlas. Roughly following the tutorial here…
lampShadesDrifter
  • 3,925
  • 8
  • 40
  • 102
1
vote
0 answers

How to display HBase data-lineage in Apache Atlas?

I am testing Apache Atlas data governance tool to display data lineage of a NoSQL database. I understand that HBase is the only supported NoSQL database as of now (input metadata source). I've set up Apache Atlas 2.0 in an environment having…
Lorem
  • 11
  • 1
1
vote
1 answer

unable to import sample Data into Apache atlas

I have installed Apache atlas using docker with the help of the below URL https://github.com/michalmiklas/atlas-docker Now while importing sample data into to apache atlas using the below command, bash-4.4# ./apache-atlas/bin/quick_start.py …
sai krishna
  • 151
  • 2
  • 11