SLA: First Response not caught

Peter_MiklianPeter_Miklian Customer Adept IT Monkey ✭✭

Our incidents have First Assigned, First Response and Resolution SLOs generated.

Sometimes when analyst adds a comment in time First Response continues to count down and is breached.

Could someone advise what could be wrong or what to check, how to detect this? Thanks.

Comments

  • Tom_HendricksTom_Hendricks Customer Super IT Monkey ✭✭✭✭✭

    The workflow that adds the first response date, in particular, is a prime offender for poor performance and for running late (and sometimes causing other workflows to run late).

    Having said that, have you checked the workflow status (Administration > Workflows > Status) to see if you have any Failed or stuck (never moves out of "Scheduled" status) workflows?

    If you do, you can either make adjustments to the batch size and timing by using a management pack override (like in SCOM) or you can disable it altogether with an override and have Orchestrator or SMA do this for you instead. They are generally much better at this than the built-in workflows.

    There could be other reasons for your issue, but this is what comes to mind first, for me.

  • Peter_MiklianPeter_Miklian Customer Adept IT Monkey ✭✭

    @Tom_Hendricks thank you for explanation.

    I checked status of every workflow in SCSM console and using this script https://scsmbuzz.wordpress.com/2017/03/30/check-for-failed-workflows/ and found only 1 failed and 9 WF instances in 'Succeeded, Need attention' state in 5 workflows. I tried to click Retry and Ignore but those 'Succeeded' didn't dissapear.

    I knew workflows are responsible for those actions but don't know which exactly does the 'record first response time and mark SLO as Met'.

    I don't like the idea of overrides, I don't want to mess with default MPs, I don't feel capable to do such inteventions.

    I already thought of having Orchestrator monitor analyst comments and change First Response SLO to Met.

    Byt which WFs could I disable then? Is there single one responsible for FR SLO? Or would I need to handle some other actions with Orchestrator?

    Btw, we are using SCSM 2016. Thanks.

  • Tom_HendricksTom_Hendricks Customer Super IT Monkey ✭✭✭✭✭
    edited July 22

    The overrides are somewhat daunting at first, until you get used to them. It is normal to see them that way, I would say.

    All you are doing, however, is setting an "enabled" attribute to "false." The kind of thing you ought to be able to do from the console, but can't. You are not changing the default MP--that would not be a good idea, and isn't even possible in many cases--you are adding your own new MP that refers to the original and says "ignore that version of it and use this instead."

    If you want to back out your change, just delete the MP. If you want to activate it again, just import it again. It is worth noting, of course, that those events will never trigger again. So you cannot go back in time if you decide you wished you had left the workflow running. So test in a lab, first!

    I have found examples of workflow override MPs out there on Google, but a (really) quick search was not getting any of the really good posts that I have used before. Having said that, something like this might work for you:

    <ManagementPack xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" ContentReadable="true" SchemaVersion="1.1" OriginalSchemaVersion="1.1">
    	<Manifest>
    		<Identity>
    			<ID>Example.DefaultWorkflow.Overrides</ID>
    			<Version>1.0.0.3</Version>
    		</Identity>
    		<Name>EXAMPLE Default Workflow Overrides</Name>
    		<References>
    			<Reference Alias="System">
    				<ID>System.Library</ID>
    				<Version>7.5.7487.0</Version>
    				<PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
    			</Reference>
    			<Reference Alias="WorkItem">
    				<ID>System.WorkItem.Library</ID>
    				<Version>7.5.7487.0</Version>
    				<PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
    			</Reference>
    			<Reference Alias="Subscriptions">
    				<ID>Microsoft.SystemCenter.Subscriptions</ID>
    				<Version>7.5.7487.0</Version>
    				<PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
    			</Reference>
    			<Reference Alias="IncidentLibrary">
    				<ID>ServiceManager.IncidentManagement.Library</ID>
    				<Version>7.5.7487.0</Version>
    				<PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
    			</Reference>
    		</References>
    	</Manifest>
    	<Monitoring>
    		<!--<Rules>
    			<Rule ID="Example.IR.FirstResponseDate.Add_Rule" Enabled="true" Target="IncidentLibrary!System.WorkItem.Incident.WorkflowTarget" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100">
    				<Category>System</Category>
    				<DataSources>
    				 <DataSource ID="DS" TypeID="Subscriptions!Microsoft.SystemCenter.CmdbInstanceSubscription.DataSourceModule">
    					<Subscription>
    					 <RelationshipSubscription RelType="$MPElement[Name='WorkItem!System.WorkItemAssignedToUser']$">
    						<AddRelationship />
    					 </RelationshipSubscription>
    					 <PollingIntervalInSeconds>100</PollingIntervalInSeconds>
    					 <BatchSize>500</BatchSize>
    					</Subscription>
    				 </DataSource>
    				</DataSources>
    				<WriteActions>
    				 <WriteAction ID="WindowsWorkflow" TypeID="Subscriptions!Microsoft.EnterpriseManagement.SystemCenter.Subscription.WindowsWorkflowTaskWriteAction">
    					<Subscription>
    					 <EnableBatchProcessing>true</EnableBatchProcessing>
    					 <WindowsWorkflowConfiguration>
    						<AssemblyName>Microsoft.EnterpriseManagement.ServiceManager.Incident.Workflows, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35</AssemblyName>
    						<WorkflowTypeName>Microsoft.EnterpriseManagement.ServiceManager.Incident.Workflows.SetFirstAssignedToDateWorkflow</WorkflowTypeName>
    						<WorkflowParameters>
    						 <WorkflowArrayParameter Name="InstanceIds" Type="guid">
    							<Item>$Data/BaseManagedEntityId$</Item>
    						 </WorkflowArrayParameter>
    						</WorkflowParameters>
    						<RetryExceptions />
    						<RetryDelaySeconds>11</RetryDelaySeconds>
    						<MaximumRunningTimeSeconds>7200</MaximumRunningTimeSeconds>
    					 </WindowsWorkflowConfiguration>
    					</Subscription>
    				 </WriteAction>
    				</WriteActions>
    			</Rule>
    		</Rules>
    		-->
    		<Overrides>
    			<RulePropertyOverride Property="Enabled" Rule="IncidentLibrary!WorkItem_SetFirstAssingedTo_RelationhsipAdd_Rule" Enforced="false" Context="System!System.Entity" ID="Example.IR.FirstResponseDate.Disable">
    				<Value>false</Value>
    			</RulePropertyOverride>
    		</Overrides>
    	</Monitoring>
    	<LanguagePacks>
    		<LanguagePack ID="ENU" IsDefault="true">
    			<DisplayStrings>
    				<DisplayString ElementID="Example.DefaultWorkflow.Overrides">
    					<Name>EXAMPLE Default Workflow Overrides</Name>
    				</DisplayString>
    				<!--
    				<DisplayString ElementID="Example.IR.FirstResponseDate.Add_Rule">
    					<Name>IR First Response Date</Name>
    					<Description>Replacement of OOB IR First Response Date workflow with larger batch size (500 instead of 150)</Description>
    				</DisplayString>
    				-->
    			</DisplayStrings>
    		</LanguagePack>
    	</LanguagePacks>
    </ManagementPack>
    


    Notice that I have commented out a couple sections. You can remove them completely, if you want to. If you imported this as-is (maybe after changing the word "EXAMPLE" to something else, too), it would only disable the IR first response date workflow. If you uncomment those sections, it will add a new IR First Response Date workflow in, with slightly different settings (I started with the original and changed the batch size).

    This article shows one good way to find the names/IDs of the OOB workflows and also how to copy the contents of the workflow if you want to start your own from it, like I did above. That information is somewhat buried near the bottom, but it is in there. There are other ways and tools to do this, too.

    Hope this helps!

  • Peter_MiklianPeter_Miklian Customer Adept IT Monkey ✭✭

    @Tom_Hendricks thank you so much for your detailed explanation, prepared MP with overrides and link for the article!

    I will read it, prepare runbook catching comments and fulfilling FR SLO and try to import your MP to stop IR FR workflow.

    I will need to handle also SR FRs. I hope I will be able to derive proper MP, too. After tests in lab I will migrate all of those to prod and let you know then, but I'll need time for all of this. Thanks.

Sign In or Register to comment.