I have a glue job that is developed in my account and i had ran it multiple times and corresponding logs are available in aws/awsglue/ log streams in cloud watch . But there are other developers also developing glue jobs, which makes me very hard to differentiate between the multiple logs names . as log names doesnt have a user friendly name in cloudwatch logs. is there a way to rename the logs related to my aws glue job ?
Asked
Active
Viewed 659 times
1
-
1The logstream prefix = the Job Run Id if that helps? So if you knew your Job Run Id you could do something like this in the cli `aws logs get-log-events --log-group-name /aws-glue/jobs/output --log-stream-name jr_....dc` or in the console – Bob Haffner Apr 30 '22 at 16:57
-
Also, I know theres a way to create custom log groups but that's if you're interested in the continuous logging for more spark info https://docs.aws.amazon.com/glue/latest/dg/monitor-continuous-logging.html – Bob Haffner Apr 30 '22 at 17:00