Home› Analyst Portal

Cireson Cache Builder

Nathan_ErdmanNathan_Erdman Customer IT Monkey ✭
in Analyst Portal
Have an issue in our Dev environment where no one can log on using Form Authentication. In the ServiceManagement DB the table CI%Domaingroup and the table CI$User are both empty. Prod has over 14000 objects combined on these 2 tables. I ran 

TRUNCATE TABLE CI$User

TRUNCATE TABLE CI$DomainGroup

TRUNCATE TABLE LastModified


Resynced AD and restarted the cache builder and both of these tables are still empty. Any ideas would be greatly appreciated.


thank you

Answers

  • Nicholas_VelichNicholas_Velich Cireson Consultant Ninja IT Monkey ✭✭✭✭
    The Ciresonportal relies on that data being populated in the ServiceManagement database, and that is done via the CacheBuilder. The first thing to check would be the CacheBuilder log to see if there are any errors present. You can find that log at C:\inetpub\CiresonPortal\bin\Logs
  • Nathan_ErdmanNathan_Erdman Customer IT Monkey ✭
    SCSM console has all the users and groups populated. heres the cachebuilder logs

    2017-07-14 08:39:10,231, ERROR [  11]:  The analyst group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 08:39:10,327, ERROR [  11]:  The knowledge manager group is invalid (DL-APP_AS_SCSM_KNOWLEDGEMGMT-ADM).
    2017-07-14 08:39:10,333, ERROR [  11]:  The announce manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 08:39:10,336, ERROR [  11]:  The asset manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 08:39:10,339, ERROR [  11]:  The announcement manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:08:44,352, ERROR [   4]:  The analyst group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:08:44,422, ERROR [   4]:  The knowledge manager group is invalid (DL-APP_AS_SCSM_KNOWLEDGEMGMT-ADM).
    2017-07-14 09:08:44,425, ERROR [   4]:  The announce manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:08:44,428, ERROR [   4]:  The asset manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:08:44,431, ERROR [   4]:  The announcement manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:53:57,478, ERROR [   4]:  The analyst group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:53:57,547, ERROR [   4]:  The knowledge manager group is invalid (DL-APP_AS_SCSM_KNOWLEDGEMGMT-ADM).
    2017-07-14 09:53:57,551, ERROR [   4]:  The announce manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:53:57,554, ERROR [   4]:  The asset manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:53:57,558, ERROR [   4]:  The announcement manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:57:18,008, ERROR [   4]:  The analyst group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:57:18,079, ERROR [   4]:  The knowledge manager group is invalid (DL-APP_AS_SCSM_KNOWLEDGEMGMT-ADM).
    2017-07-14 09:57:18,084, ERROR [   4]:  The announce manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:57:18,088, ERROR [   4]:  The asset manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 09:57:18,093, ERROR [   4]:  The announcement manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:06:38,248, ERROR [   5]:  The analyst group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:06:38,280, ERROR [   5]:  The knowledge manager group is invalid (DL-APP_AS_SCSM_KNOWLEDGEMGMT-ADM).
    2017-07-14 10:06:38,295, ERROR [   5]:  The announce manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:06:38,295, ERROR [   5]:  The asset manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:06:38,295, ERROR [   5]:  The announcement manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:29:59,104, ERROR [   4]:  The analyst group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:29:59,173, ERROR [   4]:  The knowledge manager group is invalid (DL-APP_AS_SCSM_KNOWLEDGEMGMT-ADM).
    2017-07-14 10:29:59,177, ERROR [   4]:  The announce manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:29:59,179, ERROR [   4]:  The asset manager group is invalid (SG-SCSM_DevAdmins).
    2017-07-14 10:29:59,183, ERROR [   4]:  The announcement manager group is invalid (SG-SCSM_DevAdmins).
  • Nicholas_VelichNicholas_Velich Cireson Consultant Ninja IT Monkey ✭✭✭✭
    1) Check that the groups exist in the CMDB of SCSM. You can confirm these by checking in Console -> Configuration Items -> Users. 

    2) Check that the security groups you have specified during the install of the Portal are the correct names of the security groups in Active Directory. If they are not correct, you'll need to update the values in your Setting Items table following by a restart of the Cache Builder service, or re-running the installation with the correct AD Security Group names.

     3) Check that the Distinguished Name of the Security Groups in SCSM Console against the Distinguished Name of the Security Groups in Active Directory. If the two do not match, you'll need to re-run your AD Connector, then perform the remainder of the steps as outlined in point (2) above.
  • Nathan_ErdmanNathan_Erdman Customer IT Monkey ✭
    Nicholas_Velich          


    1) The groups do exist in the SCSM console

    2) The groups do match


    3) The distinguished names match in the console and AD


    Re-synced the AD connector and restarted the Cirson cache builder with the same results
  • Brian_WiestBrian_Wiest Customer Super IT Monkey ✭✭✭✭✭
    Microsoft Support provided me with SQL commands that can force a Full AD sync on the connector. I was running into some group issues and this resolved it.

    To do a FULL AD connector synchronization you need to reset the watermark on the AD connector.  Steps below:

    -- Execute the following SQL Query to find the DataSourceID number for the AD Connector that is used to bring the users into SM

    Use ServiceManager

    Select * from LFX.datasource

    -- Update the line below replacing # with the corresponding AD connector DataSourceID from above to reset the AD Watermark 

    -- for that connector.  The Next AD Synchornization performed will be a FULL synchronization.   Will take a lot longer to run

    exec [LFX].[ResetWatermarkForDataSource] #,'erroroutput'

    From the Service Manager Console select the AD Connector and then click "Synchronize Now".  

  • Nathan_ErdmanNathan_Erdman Customer IT Monkey ✭

    Thank you Brian_Wiest

    I have already tried to clear the watermarks as well to fix this issue. SCSM sees all of my users and groups and the users and groups show that the correct datasource connector pulled them in, the problem lies in that these 2 tables CI$Domaingroup and the table CI$User  are completely empty. With out any users being in CI$User table there is no way to log in as it cannot authenticate.

  • Brian_WiestBrian_Wiest Customer Super IT Monkey ✭✭✭✭✭
    Stop the Cache builder
    Delete or rename bin\Logs\CacheBuilder.log
    Run SQL command
    TRUNCATE TABLE CI$User; 
    TRUNCATE TABLE GroupMembership_CI$DomainGroup_CI$User;
    TRUNCATE TABLE LastModified;
    Restart the Cache builder
    Monitor the log for errors. 
    If still failing you will need to up the logging level in the Cachebuilder service. 


  • Jason_MeyerJason_Meyer Customer Advanced IT Monkey ✭✭✭
    Has anyone come up with an automated tool to monitor the Cache Builder process?
  • Tom_HendricksTom_Hendricks Customer Super IT Monkey ✭✭✭✭✭
    Yes: https://community.cireson.com/discussion/3452/updated-scom-management-pack-for-monitoring-cireson-portal-and-cache-builder#latest
  • Jason_MeyerJason_Meyer Customer Advanced IT Monkey ✭✭✭
    edited June 2018

    I'm guessing that only monitors if the Cache Builder 'service' is running.   What I'm talking about is if the actual process where once every 24 hours the cache is rebuilt runs successfully or not.  We have had a few occasions where that process does NOT run successfully, the 'service' is still running and we have to take manual steps to intervene and resolve.  The Cireson Portal drop down menus stop working sometimes, the Team Work 'views' stop updating, etc, until we manually stop services, delete files, restart services.

  • Adam_DzyackyAdam_Dzyacky Product Owner Contributor Monkey ✭✭✭✭✭
    What @jason_meyer is getting at sounds doable through SCOM monitoring/looking for update to the actual cachebuilder log file.
  • Tom_HendricksTom_Hendricks Customer Super IT Monkey ✭✭✭✭✭
    @Jason_Meyer, that's correct--it monitors the service, not the integrity of the data.  I agree with @Adam_Dzyacky that this sounds possible to monitor, either by looking at the log file or via script.

    The way that I determine if something is wrong with the cache builder is to look for the individual job threads to be "late."  It is a relative term, but you can calculate it in a SQL query or script:

    ServiceManagement.dbo.LastModified![Modified] + "Refresh Interval in minutes" = "Next Run" (date/time when job should run)

    The refresh interval can be found in [drive]\InetPub\CiresonPortal\bin\Cireson.CacheBuilder.WindowsService.exe.config.

    The defaults are:
    "UsersAndGroups" - 120
    "ConfigItems" - 60
    "EnumLookup" - 1440
    "ServiceCatalog" - 1440
    "WorkItems" - 1
    "ScopedAccess" - 1440

    If you perform the calculation and the "next run" date is in the past, then there is likely a problem with the Cache Builder.  It is sometimes normal to have the jobs run behind for a minute or two if you truncated the LastModified table or performed a truncation on all the cache tables, for example, but falling behind for much longer than that should raise concern.

    So it ought to be somewhat straightforward to A.) capture this on a frequent basis, B.) calculate the next expected run, and C.) have thresholds for when the job is considered "behind."
  • Justin_WorkmanJustin_Workman Cireson Support Super IT Monkey ✭✭✭✭✭
    I like what @Tom_Hendricks said here!  Personally, I watch LastModified 'manually'.  I have a SQL table widget that just gets the whole table and if anything(particularly the WorkItem job) seems out of whack, I manually restart the service.  Having said that, this all could be automated as Tom and Adam suggest.
Sign In or Register to comment.