-1

I recently created a new production cloud composer environment and migrated my existing dags to it and have been getting the error INFO - Task exited with return code Negsignal.SIGSEGV on some tasks that work perfectly fine in my dev environment (also on cloud composer). The production environment is loaded from a snapshot of the dev environment so they should be identical. Is there anything configuration wise that might not have come over in the snapshot that I can look into so I can try and get this resolved?

For added context, I have upgraded the scheduler, web server and worker specs in the prod environment to be more than the dev environment and this is still causing the same issue.

tacoofdoomk
  • 121
  • 1
  • 1
  • 7
  • What version is your Cloud Composer for both dev and prod? What kind of tasks are you running? Can you provide your DAG as well? – Ricco D Sep 09 '22 at 03:18
  • Dev is composer 2.0.11 and prod is 2.0.25 (which seems to be the oldest version I can easily use now). The task is reading from a datawarehouse. I'd rather not post the dag code but that shouldn't really matter, it's exactly the same code in both environments – tacoofdoomk Sep 09 '22 at 12:06

1 Answers1

0

Are the instances the same size? This message means that some tasks run out of resources so you have either to scale up or modify your code so it is more frugal and you don't run out of resources.

Also have you checked how the dags are scheduled? Maybe you have many dags running at the same time and they use up all your resources.

mikrohelen
  • 249
  • 1
  • 3
  • I suspected that might be the issue and the worker specs have been increased beyond those of the dev environment and still no luck. The error remains consistent regardless of how many dags are present in the environment. I tried running it prior to migrating the other dags over and still no luck – tacoofdoomk Sep 12 '22 at 18:13