"failedLimitProcessingTime" on Design Automation Workitem

Question

I wrote a Design Automation Revit addin to export 3d geometry and material data from a RVT file. It has been working 100% in prod until last weekend.

Now when I POST a workitem, the workitem dont continue past below:

[03/15/2021 11:24:29] End download phase successfully. [03/15/2021 11:24:29] Start preparing script and command line parameters. [03/15/2021 11:24:29] Command line: [ /i "T:\Aces\Jobs\d25b06b3501c47f9b123ac282aa2e531\file.rvt" /al "T:\Aces\Applications\CLIENT_NAME.REDACTED.PACKAGENAME_REDACTED[4].package"] [03/15/2021 11:24:29] End preparing script and command line parameters. [03/15/2021 11:24:29] Start script phase. [03/15/2021 11:24:29] ### Command line arguments: /isolate HKEY_CURRENT_USER\SOFTWARE\AppDataLow\Software\Autodesk\CoreUser\WorkItem_d25b06b3501c47f9b123ac282aa2e531 "T:\Aces\Jobs\d25b06b3501c47f9b123ac282aa2e531\userdata" /exe "T:\Aces\AcesRoot\21.0\coreEngine\Exe\revitcoreconsole.exe" /i "T:\Aces\Jobs\d25b06b3501c47f9b123ac282aa2e531\file.rvt" /al "T:\Aces\Applications\c096df734873f27f2b44f3de9e7e5843.3NtdG9abSFMTefGhKAFq6IpmzdOlAcxj.EnscapeImporter[4].package". [03/15/2021 11:24:29] Start Revit Core Engine standard output dump. [03/15/2021 11:24:29] Running in unattended mode. [03/15/2021 11:24:29] ====== Revit is running: revitcoreconsole ====== [03/15/2021 11:24:29] Current Exe path: T:\Aces\AcesRoot\21.0\coreEngine\Exe\revitcoreconsole.exe [03/15/2021 11:24:29] Echoing command line args: [03/15/2021 11:24:29] 0:/i [03/15/2021 11:24:29]
1:T:\Aces\Jobs\d25b06b3501c47f9b123ac282aa2e531\file.rvt [03/15/2021 11:24:29] 2:/al [03/15/2021 11:24:29]
3:T:\Aces\Applications\c096df734873f27f2b44f3de9e7e5843.3NtdG9abSFMTefGhKAFq6IpmzdOlAcxj.EnscapeImporter[4].package [03/15/2021 11:24:29] 4:/isolate [03/15/2021 11:24:29]
5:HKEY_CURRENT_USER\SOFTWARE\AppDataLow\Software\Autodesk\CoreUser\WorkItem_d25b06b3501c47f9b123ac282aa2e531 [03/15/2021 11:24:29]
6:T:\Aces\Jobs\d25b06b3501c47f9b123ac282aa2e531\userdata [03/15/2021 11:24:29] Running RevitAssemblyResolver.... [03/15/2021 11:24:29] Initializing RevitCoreEngine... [03/15/2021 11:24:29] Selected Revit\RCE install Path: (from app.config) C:\Revit2021 [03/15/2021 11:24:29] Resolving location of Revit/RevitCoreEngine installation... [03/15/2021 11:24:29] Running user application.... [03/15/2021 11:24:33] Found an addIn for registration: EnscapeImporter.addin [03/15/2021 11:24:33] Initializing RCE.... [03/15/2021 11:24:33] Initializing RevitCoreEngine (and possibly running stub installer)... [03/15/2021 11:24:33] Language not specified, using English as default [03/15/2021 11:24:35] Initialize and get RCE: (VersionBuild) 21.1.20.44 (VersionNumber) 2021 (SubVersionNumber) 2021.2

And eventually a time out:

[03/15/2021 11:48:59] Error: Retrying on HttpRequestException. Request is 'PUT' 'https://wlnr5sjl3a.execute-api.us-east-1.amazonaws.com/Prod/v3/workitems/progress'. Exception: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory1 buffer) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.Handlers.ProgressMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates1 shouldRetryResultPredicates, Func5 onRetryAsync, Int32 permittedRetryCount, IEnumerable1 sleepDurationsEnumerable, Func4 sleepDurationProvider, Boolean continueOnCapturedContext)

I have tested the addin locally using the RVT file in question and it works as expected.

For any Autodesk devs: The workitem in question is: d25b06b3501c47f9b123ac282aa2e531