0

We have an ASP.NET Core 3.1 application running in Amazon EC2 instances with Amazon Linux 2 (RHEL based).

Periodically our application crashes with an 11/SEGV status (segmentation fault) so we enabled minidumps to be generated with an environment variable (COMPlus_DbgEnableMiniDump) as documented here

As multiple instances of the application run simultaneously within an auto scaling group, it's hard to keep track of the crashes, so I need to know if there is any tool or recommended way of logging each of these crashes and uploading the generated minidump file into an S3 bucket, so we can easily retrieve them and analyze them in our development environment.

Any recommendations?

Thank you!

Emiliano Sandler
  • 44
  • 9

1 Answers1

0

Sorry that I am late to this conversation. Hopefully, you have found a solution to this by now.

Adding my thoughts here to help anyone else with a similar challenge.

I can think of a couple of solutions:

Since the application is running on a Linux instance, you could consider saving the crash dumps to an EFS instance. Register a lifecycle hook handler to the ASG and raise an SNS notification capturing the necessary details of the crash dump file.

  • Option 1: Deploy a process as a side-car that responds to the notification and moves the dump file to S3 bucket. Please note that the dump file will be moved by the process that runs on the new instance (or other instances) spun up by the ASG
  • Option 2: Deploy the process that is responsible for moving the dump files to S3 in a dedicated EC2 instance and attach the same EFS the instance used by the actual service
  • Option 3: Create a lambda with required permissions to access the EFS access points.

Refer to AWS EC2 Lifecycle Hooks

Saman Salehi
  • 1,004
  • 1
  • 12
  • 19
Akhilesh Bangalore
  • 189
  • 5