Stack Overflow
Stack Overflow

Questions tagged [aws-glue-connection]

40 questions
0
votes
0 answers

AWS Glue Job Success But there's no data in DynamoDB Table

Everyday I try to delete and re-create the dynamoDB table, the code successfully deletes the table, but the other job fails to transfer data from Redshift to DynamoDB. The first AWS Glue job, I try to delete the table, then re-create it: import…
cey
  • 135
  • 1
  • 2
  • 10
0
votes
0 answers

AWS Glue - Is it possible to create a custom JDBC connector to WRITE into Salesforce DB

I need to do a POC if it is feasible use AWS Glue to READ data from AWS and WRITE it into Salesforce via JDBC custom connector. Checking the docs like AWS Custom Glue connectors docs :…
mafehx
  • 363
  • 2
  • 6
  • 14
0
votes
0 answers

How to create dynamic dataframe from AWS Glue 3 catalog in local environment?

I I have performed some AWS Glue version 3.0 jobs testing using Docker containers as detailed here. The following code outputs two lists, one per connection, with the names of the tables in a database: import boto3 db_name_s3 =…
GDC
  • 17
  • 6
0
votes
1 answer

How can I import external python libraries in python shell AWS Glue job

I have been trying to import an external python libraries in aws glue python shell job. I have uploaded the whl file for Pyodbc in s3. I referenced the s3 path in "python library path" in additional properties of Glue job. I also tried to give job…
srikanth sanapathi
  • 11
  • 1
  • 4
0
votes
0 answers

Is secretsmanager dynamic reference works in Glue Cloudformation?

I am trying to create a Document Db connection in Glue Cloudformation with secret manager dynamic reference ({{resolve:secretsmanager:${DBSecretManagerArn}:SecretString:username}}) During build there is no evaluation failure but before the job…
B. Patel
  • 3
  • 2
0
votes
1 answer

What is the key of Required SSL while creating Glue Connection (MONGODB connection type) in CloudFormation

i am trying to create a Glue connection using cloud formation. The connection type is MONGODB. I am not able to set the ssl details here. Tried MONGODB_ENFORCE_SSL but it's not working even though build not failed. Some-Name: Type:…
B. Patel
  • 3
  • 2
0
votes
1 answer

Not able to create dynamic frame, After crawling MONGODB table to AWS Data Catalog successfully

I created a mongodb connection successfully, my connection tests successfully and was able to use a Crawler to create metadata in the Glue Data Catalog. However, when i use below where i am adding my mongodb database name and collection name in…
Beny Mahajan
  • 3
  • 1
0
votes
1 answer

Issue developing AWS Glue ETL jobs locally using a Docker container

I am using an Apple M1 Pro Mac & trying to use a Docker container to developer AWS Glue Jobs locally and not use the AWS Console. I have been working through this blog post by AWS and I have pulled amazon/aws-glue-libs:glue_libs_3.0.0_image_01 from…
samuel
  • 23
  • 5
0
votes
1 answer

Terraform glue connection that avoids overwriting connection_properties upon apply

I have a Terraform resource for an AWS Glue Connection, like this: resource "aws_glue_connection" "some-connection-name" { name = "some-connection-name" physical_connection_requirements { availability_zone = var.availability_zone …
John Skiles Skinner
  • 1,611
  • 1
  • 8
  • 21
0
votes
1 answer

Error parsing parameter '--targets': Expected: '=', received: '{' for input:

I am creating a crawler through AWS CLI in Glue, but facing issue. Here's the CLI Command that i use aws glue create-crawler \ --name "mytestcrawlernew" \ --role "arn:aws:iam::role/AWSGlueServiceRole-AWSGlueServiceRole" \ --database-name "mydb"…
Devanshi
  • 1
1 2
3