7

I'm trying to stream data from DynamoDB to ElasticSearch. I've checked the documentation by AWS + some other sources online but I'm stuck on a security issue. So I'm using a lambda function, the process of retrieving data from DynamoDB is fine, but then when I try to write back to ElasticSearch I get an error:

"no permissions for [indices:data/write/bulk] and User [name=arn:aws:iam::account number:role/dynamodb_to_es, backend_roles=[arn:aws:iam::account number:role/dynamodb_to_es], requestedTenant=null]"

This is my Lambda function. I realized it would always fail because of the "_bulk" extension. Thank you!!

My Lambda function: https://github.com/YassineRjl/Lambda-Func---DynamoDB-to-ElasticSearch/blob/master/lambda_func.py

My IAM role: enter image description here

NoSQLKnowHow
  • 4,449
  • 23
  • 35
legacycode
  • 147
  • 2
  • 7

4 Answers4

9

You don't have to disable "fine-grained access control". Instead, you can edit role mapping.

For detailed information & steps please check:

https://aws.amazon.com/tr/premiumsupport/knowledge-center/es-troubleshoot-cloudwatch-logs/

"I'm unable to stream my CloudWatch log group to an Amazon ES domain when fine-grained access control is enabled." subject.

Alper
  • 231
  • 4
  • 11
4

Elastic Kibana Dashboard

If you're finding this from Google like me, this worked for my use-case:

  1. Open your Kibana dashboard (https://your-domain-somerandomstring.us-east-1.es.amazonaws.com/us-east-1.es.amazonaws.com/_plugin/kibana/app/opendistro_security#/roles/view/all_access/mapuser)
  2. Navigate to Security in the left nav
  3. Select Roles
  4. Select a role that you'd like to attach your user to (in the example above, it's "all_access")
  5. Add your Username from the error log, or use the ARN for your username (mine was related to the instance profile associated to the instance I was connecting from)
B Fish
  • 323
  • 2
  • 7
  • 1
    I believe only the user is required in the mapping (providing the internal user name and backendrole are both the ARN of the IAM principal). I haven't tested skipping the user and using the ARN directly against the role, but that might work too. – Chris Austin Nov 30 '21 at 10:44
2

I found the solution. During the creation of the ES instance, make sure to unselect "fine-grained access control" & avoid VPC for the sake of Https, then on the roles, create a role on IAM and copy-paste the ARN in the ES dashboard during the instance setup

legacycode
  • 147
  • 2
  • 7
0

You should map your user to role: kibana_user, which defines basic permission to access index.

李泽鹏
  • 11
  • 1