3

With the same exact Data Pipeline configuration, only differing in the AMI to be used (Amazon Linux vs. Ubuntu), my Data Pipeline execution will succeed in both cases but it will only write logs to S3 when using Amazon Linux.

With Amazon Linux

enter image description here

With Ubuntu

enter image description here

In both cases I login with the same user (ec2-user, not ubuntu), for which I properly configured that username for the Ubuntu AMI:

#cloud-config
system_info:
  default_user:
    name: ec2-user

Moreover, I use the same exact resourceRole and role attributes when launching Amazon Linux or Ubuntu pipelines. So that's not the issue.

So apparently Amazon Linux has something needed for writing S3 logs, what could it be?

deprecated
  • 5,142
  • 3
  • 41
  • 62

1 Answers1

2

This is because TaskRunner uses a Java library called Joda to generate timestamps for the logging. Some versions of the JRE ship with a buggy version of the Joda jar, so any AMI that uses that version (anything above 6, in my experience so far) won't be able to write logs correctly.

I'd recommend including something like alternatives --set java /usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java before all your scripts in the ShellCommandActivity -- this fixed the problem for me.

Alternately, you could just always use an AMI instance ID that is known to have Java 6 on it.

shrikant
  • 951
  • 6
  • 9
  • Wow I would have NEVER imagined that could be the issue! :) I use Oracle JRE `8u65-b17`, which is the latest. Is that version known to have that bug? Should I try Java 7 instead of 8? – deprecated Nov 20 '15 at 13:20
  • If you're using anything beyond `8u65`, you should ensure that your version of Joda is updated to something newer than `2.8`. You could also downgrade to anything below `8u45`, so this presumably includes Java 7, although I've only really tested by downgrading to Java 6. See [this for a greater level of detail](http://stackoverflow.com/questions/32058431/aws-java-sdk-aws-authentication-requires-a-valid-date-or-x-amz-date-header). – shrikant Nov 20 '15 at 15:03
  • Using Oracle `7u80-b15` made it work. Thanks so much! – deprecated Nov 20 '15 at 15:21
  • Is this answer still valid? Does ec2 instance still have java 6? I am not able to get any logs for a t2.micro instance – Joy Chopra Dec 24 '19 at 14:33