Stack Overflow
Stack Overflow

Questions tagged [aws-lake-formation]

79 questions
15
votes
0 answers

AWS Lake Formation AccessDenied

I've enabled Lake Formation and tried to create a Database but no matter what I still get an error: Insufficient Lake Formation permission(s): Required Create Database on Catalog I'm using root account with all privileges. I even gave…
Vangeth
  • 151
  • 1
  • 3
11
votes
2 answers

What are the major differences between S3 lake formation governed tables and databricks delta tables?

What are the major differences between S3 lake formation governed tables and databricks delta tables? they look pretty similar.
MGomez
  • 123
  • 1
  • 5
10
votes
3 answers

AWS Lake Formation: Insufficient Lake Formation permission(s) on s3://abc/

I'm trying to setup a datalake from AWS Lake Formation Console. I'm following the below resource to do the same: https://aws.amazon.com/blogs/big-data/integrating-aws-lake-formation-with-amazon-rds-for-sql-server/ But in second step while creating…
Jake
  • 391
  • 1
  • 4
  • 22
4
votes
0 answers

AWS Lake Formation, Can we add LF tag to column in nested JSON?

Ex. 1 table with 10 columns, out of which 1 column is structarray:>> With multiple levels of structs and arrays inside them. Can we hide "Name" column by adding LF tag to it? Tags are already defined and assigned to…
Saipraasad
  • 53
  • 4
4
votes
1 answer

AWS Quicksight not able to see athena db

I'm grasping at straws at this point. I have a Database in LakeFormation and I've given the quicksight service role access to it as well as underlying tables (and their S3 buckets). I've also verified that the quicksight location (us-east-1 N.…
de1337ed
  • 3,113
  • 12
  • 37
  • 55
3
votes
0 answers

Unable to grant cross-account access to Glue tables via Lake Formation

I'm trying to provide cross-account Glue access to Account B from Account A. I'm first getting an error that says, User {my_arn} is not authorized to perform: glue:GetDatabases on resource: {catalog} I researched and found that I can grant Data…
Ricardo Francois
  • 752
  • 7
  • 24
3
votes
1 answer

Usage of AWS Lake Formation with CloudFormation

I want to set up an additional security layer on top of my S3 / Glue Data Lake using Lake Formation. I want to do as much as possible via Infrastructure as Code, so naturally I looked into the documentation of the CloudFormation implementation of…
Robert Kossendey
  • 6,733
  • 2
  • 12
  • 42
3
votes
1 answer

Is crawler required for creating an AWS glue job?

I'm learning Glue with Pyspark by following this page: https://aws-dojo.com/ws8/labs/configure-crawler/. My question is: is crawler & creating a database in Lake Formation required for creating a glue job? I have some issue with my aws role and I'm…
wawawa
  • 2,835
  • 6
  • 44
  • 105
3
votes
1 answer

Lake Formation Governed Table underlying format/technology

Lake Formation anounced preview for ACID and RLS features. In the nearest future the next step towards Lakehouse architecture would be possible on EMR+LakeFormation without extra management layer like Databricks. What data format/technology is used…
VB_
  • 45,112
  • 42
  • 145
  • 293
3
votes
1 answer

AWS Lake Formation : grant_permissions : Unknown parameter in Resource.Table: "TableWildcard"

Trying to grant lake permissions via a Lambda Function. (Python 3.8) As far as I can see, I have my code as per documentation. Yet hitting a barrage of nonsense errors about parameters being incorrect. Could it be that I just need an optician ? Or…
SimonB
  • 962
  • 1
  • 14
  • 36
3
votes
1 answer

AWS Glue Development Endpoint Not Working properly

I am trying to use a development Endpoint to interactively run and edit ETL scripts but there seems to some issues in the development endpoint just after creating it as i am getting errors in scala/python REPL and also unable to do SSH tunnel to…
gashu
  • 863
  • 2
  • 10
  • 21
3
votes
0 answers

AWS Glue Crawler creating empty tables in Lake Formation

I have successfully ingested data from a MySQL RDS database to S3 buckets with a Lake Formation blueprint. Upon inspecting the data, approximately 41/60 tables have been correctly ingested. Bug searching has revealed two things: My blueprint…
editworthy
  • 81
  • 1
  • 3
3
votes
2 answers

Traditional Data Lake vs AWS Lake Formation

I have been setting up data lakes for clients wherein we load the data from onprem or any other sources, into the S3 (a data lake). We will create an AWS Glue catalog on these raw data to create schemas. The next step would be to either use an EMR…
Yuva
  • 2,831
  • 7
  • 36
  • 60
2
votes
0 answers

CDK Granting Permissions in Lake Formation

I am trying to create Glue database and grant permissions on it in Lake Formation. I tried several ways and several IAM roles and policies based on the documentation but every time I get Insufficient Lake Formation permission(s): Required Create…
Moki
  • 199
  • 1
  • 4
  • 15
2
votes
2 answers

AWS Lake Formation: Grant permission for one role to ALL databases

Is it possible to somehow grant permissions to ALL available databases in Lake Formation? Using the AWS UI I'm able to grant permission to individual databases. However I'm looking for a way to grant permissions to all available databases. Is that…
plazmakeks
  • 161
  • 1
  • 10
1
2 3 4 5 6