Portal performance tuning

Roland_Kind

Hello

what activities (configuration settings) can be done, to increase the performance of the Portal ?

(e.g. IIS Settings, Browser Settings, SCSM/SQL/Portal architecture design considerations, SQL Tuning etc. ?)

I am asking, because it looks like (at least in my testing environment) that I loose some performance when opening workitems and can't figure out whats the root cause for this. (with F12 developer tools, I can see sometimes a very long "download time" or even long DNS resolution timinig which is totaly strange for me - and I don't believe its a DNS problem)

For testing, I removed all customizations included in custom.js - but the effect on this is not so high. It looks like, that sometimes the IIS is not responding very fast, but the IIS itself has no real workload.

The environment is SCSM2016/SQL2016 on Windows Server 2016 - The client is W10 with IE and Edge - The portal version is 8.2.x

The webconsole.log and cachebuilder log don't show any significant errors.

Would be great to share some tipps and tricks about that.

regards

Roland

Tom_Hendricks

I would like to see some portal tuning suggestions here, too.  File compression, etc.?  What about reducing the number of all those loadScript calls that many of us have in our customizations?  There is much I still have to learn here and I am looking forward to other responses.  But I definitely had some thoughts come to mind upon reading this.  These are more focused on the occasions where the portal is reaching back to SCSM.

Typically, these slowdowns that you see in the "Network" section of the dev tools is due to the portal communicating back to the SCSM DAS service.  Some things can be queried from Cireson's ServiceManagement DB, and they tend to run fairly quick, while others have to communicate with the local management server's DAS.  Templates are queried directly from the DAS, and ticket Open and Save actions interact directly with the DAS.  These actions tend to run slow.

If your experience is lining up with this, then you need to start looking at your ServiceManager database and at SCSM in general.  Some key areas to look at are:

• Where is your CacheBuilder running?  I just moved mine off the web server and onto my primary management server, now that it is finally running well enough to handle additional tasks.  The CB would occasionally cause slow performance during some of its syncs, and that has been far less noticeable on the primary management server than it was for users.
  
• Have you disabled ECL logging?  ECL murders performance, but is required for certain functionality (Desired-state Config Management is one such thing, if I am not mistaken) so weigh the pros and cons carefully before disabling.
• How many tickets are in your ServiceManager DB?  Is it more than what you expect, based on your grooming settings?  It is easy for tickets to hang around in your DB when you expect them to have been groomed out, due to the last modified getting changed each time you import certain management packs, for example.  I have this issue in my environment and am working with Microsoft to carefully remove those tickets.  A quarter-million tickets will take a toll on opens, saves, and history loads, for example.
  
• What restrictions are placed on your user roles?  Anything other than allowing access to everything for templates, queues, views, etc. means that the role is scoped.  This is certainly not a bad thing, but it carries a performance penalty for all members.  If you have an Administrator (never scoped) and a member of a scoped user role perform the same activity, the admin will always get a faster response.  Making everyone an admin is obviously not right, but restrictions should be strategic and few for best performance.
  
• What queues have you placed on your tickets and/or config items?  Just like restrictions on user roles, this scopes access to these items, which requires a much slower DB scan than unscoped access.  Similar to user roles, queues are not bad and may even be necessary at times, but be stingy with their use.
• If you have certain object classes in your environment that all members of a role could safely have access to (perhaps Users and Groups has no restrictions or privacy concerns for your enviornment), you can explicitly grant unscoped access to that class using a powershell script provided by Microsoft in the Service Manager section of docs.microsoft.com.  I believe this setting overrides any queues that were meant to restrict access to it, so be careful with this.
  
• How many workflows (including mail subscriptions) are you running?  This directly impacts your primary management server more than your secondaries, but a poorly running workflow server can have cascading effects throughout the management group.  Less is more, when it comes to workflows.  Orchestrator/SMA might be a better home for some automation, but the connection(s) that they make still consume resources (less than the workflow they replace, potentially).
  
• Look at your database.  Are there indexes you could apply to the ServiceManager DB?  Also, for whatever reason, SCSM performs worse with the newer cardinality estimate calculations used by SQL Server since SQL 2014.  Your compatibility level should be set to SQL 2012 (110) and you should either turn on Legacy Cardinality Estimation for the DB or set the corresponding trace flag for the whole server.  I would be interested to know what effect the server-wide setting has on the Cireson databases. 
  

There are no doubt more areas you could look at, and I also wonder if anyone else has had a different experience with some of the items I suggested above.  Trying to identify and resolve these performance issues seems more like magic than science, most of the time.

Marek_Lefek

We have the same issue since sime time.

Jerry_Veldhuis

Keep your eye on the Network tab in the chrome console. After upgrading from V7.4 to V8.6.1 we noticed that custom code loaded via $.getScript() were having their urls adjusted with "?_=<timestamp>" which was triggering chrome to ignore the max-age hints we added to the CustomSpace/web.onfig file.

Instead, we adopted https://community.cireson.com/discussion/comment/14268#Comment_14268 which fixes this issue and also allows you to control which pages your customizations are loaded on.

Marek_Lefek

We have the same situation and after analysis , we've noticed that our GPO rules (eg. modifying IE settings) slows the portal reponse. The amount of groups, quees have enought influence than GPO. Also using Firefox and Chrome was 3-4 times faster than IE. 
The fastest time for opening SR, IR, was 2-4 seconds in Chrome. In IE about 5-7seconds.