Anyone else having issues with PowerShell Activities getting "stuck" in progress?

Raul_Noa

Occasionally, PSAs will hang indefinitely in an "In Progress" state. As a workaround, we find the ones that have been in progress for over an hour and set the status to "Rerun" and they always successfully execute on a subsequent run, usually in less than a minute. I've found these two variations of errors in the PSA logs for the ones that get stuck in progress, and corresponding errors in the workflow server's windows application event log.


Unexpected Error 1:  (most predominant for stuck PSAs)
======================================

3/28/2018 7:55:12 AM Invoke PowerShell Object

3/28/2018 7:56:56 AM PowerShell finish executing

3/28/2018 7:56:56 AM Printing PowerShell output to Output property

3/28/2018 7:56:58 AM Error excuting PowerShell

3/28/2018 7:56:58 AM The specified value did not conform to the maximum length requirements.
======================================
Corresponding Windows Event Logs on Workflow Server:

======================================

A Windows Workflow Foundation workflow failed during execution.   Workflow Type: Cireson.Powershell.Activity.Workflow.RunPowershellScript  Workflow Identifier: 81fd4086-4496-d1b8-1657-5dff1f7e25f0  Exception Type: Microsoft.EnterpriseManagement.Common.InvalidSimpleObjectValueException  Exception Message: The specified value did not conform to the maximum length requirements.   Exception Stack:    at Cireson.Powershell.Activity.Workflow.RunPowershellScript.a(Object A_0, EventArgs A_1)    at System.Workflow.ComponentModel.Activity.RaiseEvent(DependencyProperty dependencyEvent, Object sender, EventArgs e)    at System.Workflow.Activities.CodeActivity.Execute(ActivityExecutionContext executionContext)    at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(T activity, ActivityExecutionContext executionContext)    at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)    at System.Workflow.Runtime.Scheduler.Run()

======================================

Unexpected Error 2: (less common, but also seen with stuck PSAs)
======================================

3/28/2018 10:09:38 AM Invoke PowerShell Object

3/28/2018 10:11:18 AM PowerShell finish executing

3/28/2018 10:11:18 AM Printing PowerShell output to Output property

3/28/2018 10:11:18 AM Error excuting PowerShell

3/28/2018 10:11:18 AM Exception of type 'System.OutOfMemoryException' was thrown.

======================================
Corresponding Windows Event Logs on Workflow Server:

======================================

Application: MonitoringHost.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: System.OutOfMemoryException

Stack:

   at System.Workflow.Runtime.Hosting.DefaultWorkflowSchedulerService+WorkItem.Invoke(System.Workflow.Runtime.Hosting.WorkflowSchedulerService)

   at System.Workflow.Runtime.Hosting.DefaultWorkflowSchedulerService.QueueWorkerProcess(System.Object)

   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()

   at System.Threading.ThreadPoolWorkQueue.Dispatch()

======================================

Faulting application name: MonitoringHost.exe, version: 7.1.10184.0, time stamp: 0x522a23d5

Faulting module name: KERNELBASE.dll, version: 6.3.9600.18895, time stamp: 0x5a4b1cf7

Exception code: 0xe0434352

Fault offset: 0x00000000000092fc

Faulting process id: 0x56f4

Faulting application start time: 0x01d3c60364f18230

Faulting application path: \Program Files\Microsoft System Center 2012 R2\Service Manager\MonitoringHost.exe

Faulting module path: C:\Windows\system32\KERNELBASE.dll

Report Id: 53c029e6-31f7-11e8-811b-005056aa6011

Faulting package full name: 

Faulting package-relative application ID: 

======================================


By contrast:

Normal and Expected Error: If the script actually executes and produces an error we see a different type of error, as expected and the PSA changes to a Failed status.
======================================

3/28/2018 10:06:38 AM Invoke PowerShell Object

3/28/2018 10:06:38 AM PowerShell finish executing

3/28/2018 10:06:38 AM Printing PowerShell output to Output property

3/28/2018 10:06:38 AM Assigning ManuallyRunWorkflow to false

3/28/2018 10:06:38 AM Powershell has error.

Cannot bind argument to parameter 'Id' because it is null.Cannot bind argument to parameter 'BySource' because it is null.

3/28/2018 10:06:38 AM Start updating instance

3/28/2018 10:06:38 AM Instance successfully updated
======================================




I do have a support ticket open with Cireson, just wanted to see if anyone else is experiencing this.

Thanks!