Notify Analyst workflow intermittently not running

salzik

We upgraded to Notify Analyst 2016 but have seen a few intermittent issues.  What we're seeing is that the Notify Analyst Workflow executes correctly but once in a while the workflow does not run when it should.  I've found cases where the same SR template, with the same defined analyst, was used to create two SRs.  A notification was sent for the first SR, but not for the second SR.  When I check the Workflow > Status for the Cireson Notify Analyst Workflow, I see the record for the first SR, but no record for the second SR.  Since there is no record of the workflow running for the second SR, I'm thinking the issue must be with whatever is supposed to be triggering the Notify Analyst workflow. Can someone help me understand what workflow is supposed to trigger the Notify Analyst Workflow.

Tony_Collett

Do you have any log files that you can attach? Log files can be set up in the settings for Notify Analyst. 
Also, are there any failures in the Workflow Status under either tab?

salzik

I don't see any logging options for the free version of Notify Analyst.  There aren't any failures in the Workflow Status, so I don't think the problem is with Notify Analyst.  I'm thinking the problem is with whatever is supposed to trigger it.

Conner_Wood

Perhaps this could help you solve your issue?
Workflows aren't getting created/run AND SCSM Console - Administration - Workflows - Status is SLOW!

salzik

Conner_Wood, Thanks for that suggestion.  I already went through that post, but our workflows are not failing.  The workflow is just not always triggering.  The fact that it intermittently runs is what makes it tricky.  When it does run, it always runs successfully.  None of the other built-in workflows are failing either, otherwise, I would investigate the failing ones in case there was some dependency there.

salzik

We opened a ticket with Microsoft and they were saying that the Cireson workflows may be running faster than the built-in SCSM workflows.  This would then result in Cireson missing some tickets due to the watermarks not being synchronized.  They said there may be a way to delay the execution of Cireson workflows to make sure they don't execute faster than the SCSM ones.  Does anybody know how I can do that? 

Conner_Wood

They say there may be a way but what that's telling me is they don't know if that SCSM feature exists....

I think what they are referring to is creating a Management Pack Override (which is a management pack).  
But I don't think there's a value for a start delay... there is a tutorial on how to disable a workflow via override.

I exported the "Cireson Notify Analyst Workflows" management pack and do not see any delay property that could be overridden.  I do not like those odds.

I remember reading something about changing the EnableBatchProcessing to false or remove it on workflows since it could mess things up, but could not find documentation on it.....

I recommend your next step be this:  Workflow Performance Troubleshooting Queries

If that doesn't work, you might be able to see if there is a delay option if you could query SCSM for the override parameters on a rule/rule configuration.... but I don't think there is.

SDK DLL:  Microsoft.EnterpriseManagement.Core.dll
NameSpace:  Microsoft.EnterpriseManagement.Configuration
Class:  ManagementPackRuleOverride

https://docs.microsoft.com/en-us/previous-versions/system-center/developer/hh964103(v%3dmsdn.10)
https://docs.microsoft.com/en-us/previous-versions/system-center/developer/bb960505%28v%3dmsdn.10%29
https://docs.microsoft.com/en-us/previous-versions/system-center/developer/bb464935%28v%3dmsdn.10%29

Update:  I am going to say hell no there is not since there is no OverrideAble Parameter Name that has "Delay" in it besides "RetryDelaySeconds" when I queried the SCSM Live Database directly:

SELECT [OverrideableParameterId]
      ,[OverrideableParameterName]
      ,[ModuleTypeId]
      ,[MonitorTypeId]
      ,[ManagementPackId]
      ,[Xpath]
      ,[OPType]
      ,[Comment]
      ,[TimeAdded]
      ,[LastModified]
  FROM [ServiceManager].[dbo].[OverrideableParameter] (nolock)
  ORDER BY [OverrideableParameterName]

Justin_Workman

There is a delay setting for Notify Analyst in the Settings.  Maybe try something like 40000 milliseconds?

salzik

Justin, I don't see that option.  Do you know if that option is available in the Free version of Notify Analyst?  We're also at version 8.0.2.2016, so not sure if the newer version has that option or if I need the paid version.

Justin_Workman

@salzik - Oh sorry.  That delay is not available in the Free Version.