0

As per the doc, I am trying to create a batch job from Java Code. I am able to create a job from console with same role and lambda arn, but from code, I am getting 400 Bad Request. Also, I don't see any error message as per this doc

Here is my code snippet -

JobOperation jobOperation = new JobOperation().withLambdaInvoke(new LambdaInvokeOperation()
                    .withFunctionArn("arn:aws:lambda:eu-west-1:<account_id>:function:s3BatchOperarationsPOCLambda"));

            JobManifest manifest = new JobManifest()
                    .withSpec(new JobManifestSpec().withFormat(JobManifestFormat.S3InventoryReport_CSV_20161130)
                            .withFields(new String[] { "Bucket", "Key" }))
                    .withLocation(
                            new JobManifestLocation().withObjectArn("arn:aws:s3:::<bucket_name>/manifest.csv")
                                    .withETag("e55392fa1ad40a08e40b13b3c000a0aa"));

            JobReport jobReport = new JobReport().withBucket(reportBucketName).withPrefix("testreport")
                    .withFormat(JobReportFormat.Report_CSV_20180820).withEnabled(true).withReportScope("AllTasks");

            AWSS3Control s3ControlClient = AWSS3ControlClientBuilder.standard().withRegion(Regions.US_WEST_1).build();

            String roleArn = "arn:aws:iam::<account_id>:role/S3-Batch-Role";
            String accountId = <account_id>;
            s3ControlClient.createJob(new CreateJobRequest().withAccountId(accountId).withOperation(jobOperation)
                    .withManifest(manifest).withPriority(12).withRoleArn(roleArn).withReport(jobReport)
                    .withClientRequestToken(uuid).withDescription("S3 job").withConfirmationRequired(false));


        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process
            // it and returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            System.out.println("test2" + e.getMessage());

            // Amazon S3 couldn't be contacted for a response, or the client
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }

Role has full IAM and s3 batch operation permissions, also lambda has access permission for s3. Trust policy is also defined for batch operations.

Here is my error log -

 (Service: AWSS3Control; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; Proxy: null)
com.amazonaws.services.s3control.model.AWSS3ControlException: null (Service: AWSS3Control; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; Proxy: null)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1811)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1395)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1371)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
    at com.amazonaws.services.s3control.AWSS3ControlClient.doInvoke(AWSS3ControlClient.java:1532)
    at com.amazonaws.services.s3control.AWSS3ControlClient.invoke(AWSS3ControlClient.java:1499)
    at com.amazonaws.services.s3control.AWSS3ControlClient.invoke(AWSS3ControlClient.java:1488)
    at com.amazonaws.services.s3control.AWSS3ControlClient.executeCreateJob(AWSS3ControlClient.java:265)
    at com.amazonaws.services.s3control.AWSS3ControlClient.createJob(AWSS3ControlClient.java:236)
    at com.code.platformintegrationsscheduler.handlers.test.createS3Job(test.java:68)
    at com.code.platformintegrationsscheduler.handlers.test.main(test.java:27)
GAGAN BHATIA
  • 591
  • 5
  • 17

2 Answers2

1

I was stuck with the same issue today and after some debugging and trying out the same operation on CLI, I found that new JobReport().withBucket(reportBucketName) takes a bucketArn instead of a bucket name.

The actual issue might be different in your case. I suggest you serialize your request from code and try out the same operation in CLI and match both the requests.

AWS Error messages are often not very helpful when we actually need them.

zer0
  • 11
  • 2
  • Updated name with Arn for the bucket, doesn't seem to be working. To add to this point, I cant create batch from CLI also. Will add cli command as next comment – GAGAN BHATIA Aug 19 '20 at 05:56
0

I got the issue, issue was related to the gradle versions, we need to make sure we have all aws services gradle versions to be same.

In my case -

    compile group: 'com.amazonaws', name: 'aws-java-sdk-dynamodb', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-iam', version: '1.11.844' 
    compile group: 'com.amazonaws', name: 'aws-java-sdk-events', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-s3', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-batch', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-s3control', version:'1.11.844'
GAGAN BHATIA
  • 591
  • 5
  • 17