(v8.x) All ODATA (platform) calls fail with 401. Is this a setup/config issue?
Tom_Hendricks
Customer Super IT Monkey ✭✭✭✭✭
An example is the "Asset Analysis" dashboard. Users (including highly privileged) are prompted for credentials, but providing them does not work. Direct calls to the platform api (postman, Excel, PowerShell) also fail with a 401/Not Authorized.
The platform service runs under a credential with more than sufficient privileges, it appears (same as CacheBuilder). I see nothing wrong in IIS, also.
Is there something else I should look at before submitting an Incident?
The platform service runs under a credential with more than sufficient privileges, it appears (same as CacheBuilder). I see nothing wrong in IIS, also.
Is there something else I should look at before submitting an Incident?
0
Best Answers
-
Tom_Hendricks
Customer Super IT Monkey ✭✭✭✭✭
I can update this from my end, as I have a few observations to share, now. As of 8.2.2 / 8.3.1, this is working in my environment, but I am not entirely sure if it was a software change. Perhaps that is part of it.
It seems to matter very much whether your environment is configured to allow connections to /platform/api, BEFORE the installer runs. In other words, installing the portal then setting up your server to conform to all this endpoint's demands later does not seem to work.
I had a dev environment and a staging environment, both on portal v8.2.2, both with identical IIS configs. Dev worked, staging did not. That seems to be because we added an SPN and configured DTC in dev first, then rolled those changes to staging later, which happened to be after the portal was upgraded in both environments. I just now updated both environments to portal 8.3.1 and it fixed the issue in staging, while dev continues to work properly.
The DTC config I am referring to was mentioned by @John_Long in another thread (https://community.cireson.com/discussion/comment/10723#Comment_10723 ).
0 -
seth_coussens
Member Ninja IT Monkey ✭✭✭✭
We have some updates coming for this in 8.4.1 as well around authentication.
DTC hasn’t been required since v8 though so that shouldn’t be necessary.5
Answers
Tried to use the new shiny https binding in the Cireson setup but that made the installation fail so I tested one more time but this time with only http (port 80) and everything seemed to go smooth. Immediately Changed the binding to https (hostname) and removed the http as a have a redirect on port 80 in the default site to https://hostname:443
After this whenever and however I tried to access the platform api I only got a 404 Not found. Found the Platform folder in CiresonPortal with a 'Platform_Cache.config' where all settings is and in this case it's pointing to port 80 as that's what I had to use to be able to upgrade to v8.x
Changed the bindings and restarted the services/pool/web and I was finally able to access the OData endpoint on port 80. Did test the table and chart again but was on https so I could be the reason it didn't work. Aside exploring the endpoint here's where my testing ended.
Don't know why but in my case the Platform service was using the App pool account which doesn't have any privileges in SCSM db and I saw in the log.txt in the Platform folder repetitive errors with the sql client trying to access that db so being not even a test environment I went the easy route and made that account dbowner.
Really need to take some time and get to the bottom with this!
Regarding the OData endpoint (which I absolutely love that Cireson taken this route) the only sad part is that it's not for WI, which would have opened up a whole new world of smooth custom possibilities...
Your walkthrough did prompt me to look in the platform folder again--somehow I did not notice the log.txt file. In it, I find many repetitions of this error, which may indicate a different problem:
I find myself wondering if there needs to be a CDATA wrapper in an XML file somewhere, or escaping of strings/characters in other types of files, for this to work in the wild. I am thinking of my recent SMA Connector post, too.
Don't know if the https binding is totally broken or if it works if you remove the current binding before running the installer. Do remember some complaints about my SSL cert but didn't check the install log before re-running the installer. The idea here is to manage to get it installed with https and thus get the correct settings in the .config file. Did try to update the config file with correct protocol, port and cert thumbprint but was never able to access the odata endpoint on https.
Again, the libraries may be choking on unexpected characters in my case, but it refuses to return anything other than 401 over here. I hope other folks are having better luck with it.
When installing the update to v8.1.0 in my dev environment trying to enable https I got the same error related to my certificate.
<div>8/4/2017 2:37:52 PM Installing Platform Cache Service <span style="background-color: transparent; color: inherit; font-size: inherit;"><font face="Helvetica Neue, Helvetica, Arial, sans-serif">8/4/2017 2:37:52 PM Failed to install windows service: parsing "*.domain.com Alpha" - Quantifier {x,y} following nothing.</font> </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:37:52 PM Failed to install windows service: parsing "*.domain.com Alpha" - Quantifier {x,y} following nothing. </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:38:35 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:38:45 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:38:55 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:39:05 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:39:15 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:39:25 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:39:35 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:39:45 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:39:55 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:40:05 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:40:15 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:40:25 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:40:35 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:40:45 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:40:55 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:41:05 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:41:15 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:41:25 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:41:35 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:41:45 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:41:55 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:42:05 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:42:15 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:42:25 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:42:35 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:42:45 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:42:55 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:43:05 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:43:15 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:43:25 PM Platform not ready yet, please wait... </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;">8/4/2017 2:43:35 PM Unable to initialize Platform</span></div>Somehow the installer isn't able to parse the friendly name of my certificate (Alpha is the CA thats why i'm using it in friendly).
Changed from *.domain.com Alpha => wildcard.domain.com and re-run installer without any problem or errors. The OData endpoint works as supposed but I can still see that the App Pool service account is used for the Platform cache and I already made the necessary security arrangements for that account.
Do you have any luck if you do not include a certificate in the installer, then add the 443 binding after the portal is installed?
I have a few post-install steps that I have a powershell script for, and one of them used to be setting up the 443 binding that the installer used to wipe out. What I am not sure about is whether leaving this out of the install process may have some detrimental effect.
That part of the PS script, in case it is helpful:
$PortalSiteName = "CiresonPortal"<br>$CertFriendlyName = "FRIENDLY NAME OF YOUR CERT"<br><br># Get the certificate, to be used when re-binding SSL after portal installation<br> $Cert = Get-ChildItem Cert:\LocalMachine\My | ? { $_.FriendlyName -like $CertFriendlyName }<br><br># Navigate to the IIS Namespace after loading the module<br> if ($(Test-Path IIS:\)) {<br> Set-Location IIS:\<br> }<br> else {<br> Throw "ERROR: IIS Path not available. Check WebAdministration module and try again."<br> }<br><br> # Restore the bindings AFTER the portal update - test to see if the https binding exists first, because http always will<br> if (-Not (Get-WebBinding -Name $PortalSiteName | ? { $_.protocol -eq "https" })) {<br> Write-Output "SSL Binding removed by Cireson Installer. Restoring Bindings with certificate..."<br> <br> try {<br> Set-Location IIS:\SslBindings<br> New-WebBinding -Name $PortalSiteName -IPAddress * -Port 443 -Protocol "https"<br><br> Write-Output "SSL Binding restored."<br> }<br> catch {<br> Write-Output "ERROR: SSL Binding was not restored! Please add the binding and certificate back to the site manually. Error Details: $Error"<br> }<br> try {<br> #separate this from the try/catch above, because sometimes an error is generated if the binding needed to be restored but the cert is already assigned.<br> $Cert | New-Item 0.0.0.0!443<br> }<br> catch {<br> Write-Output "**Potential** Error: The cert could not be added to the SSL binding. This could be because it already exists. Review the following error message for further instruction: $Error"<br> }<br> }<br> else {<br> Write-Output "SSL Binding was not removed by installer, so no action was required."<br> }<br>This always ends up at the final "else" now, thanks to the addition of the certificate in the newer installers. I still have to change the session timeouts on the app pool, re-establish http/https redirects, add back virtual directories and virtual apps, of course.
Are we using the SSL thumbprint when we are inputing the SSL certificate or are we using the name? Can someone post their Platform.Cache.config file as well as their log file in the programdata/platform.core folder?
{"HostWebServer":true,"Urls":["http://*:80/Platform","https://*:443/Platform"],"WorkerCount":1,"ConnectionString":"Data Source=<b><i>(NAMEOFDATABASESERVER)</i></b>;Initial Catalog=ServiceManagement;Integrated Security=True;Connect Timeout=15;Encrypt=False;TrustServerCertificate=False;Trusted_Connection=True","ShowHelp":false,"Basic":false,"SslCertificateThumbprint":"<b><i>(CERTHUMBPRINT)</i></b>","ProductKey":"","MasterExtensionName":"PlatformCache","IsRunningAsService":false}There are a few parts of this that look suspicious right away. Why doesn't ProductKey have a value? Why isn't it running as a service (I have a "Platform" service running now, as of v8, after all)? Why is Encrypt set to "false"?
My log file is almost 20 MB, and I cannot post it publicly until/unless I can take the time to be 100% certain that there is nothing in it that is unique to our environment, but I can send it over to you privately. For right now, here are some items that jump out to me among the many errors:
I have no idea why this would occur, but this message is extremely prolific in the log. But I am also not sure which server is supposedly not responding? The web server (which this is running on) or the database server? There should not be any issues on the DB server, unless this is using a port that is blocked or a service such as DCOM, WMI, WinRM, etc. that is being actively blocked. The account that this runs on has all the authorization it needs to that database server. We do have a lot of blocked queries and deadlocks right now due to performance issues, but this fails 100% of the time, and nothing else comes close to that. In sharing about 25GB of logs with Microsoft, they identified which tables are causing the blocked and deadlocked queries, and it is primarily the ECL and RelatedECL tables. None of the Cireson tables showed up. (Unrelated side note: the Cireson KB Full Text search stored procedure did show up as the #1 operation with the most unindexed seeks/scans, though. Yes, more than the ECL tables!)
I do not see very many of these, and could easily see where this is just due to those performance issues I described above. However, I am not quite sure how it could even get to the point where a deadlock is possible if it cannot even connect in the first place.
Circling back to the first error, here is the rest of the message, which has some file references (in a container??) and line numbers that may be helpful:
Definitely looks like a connectivity problem, though could be related to configuration rather than real network connectivity issues. Have you opened a ticket for your issue as well? If so what is the IR#? Also, what version of the portal are you currently using?
I agree with your take on this, I think something is artificially blocking this, but I have no sense of what that might be--just what it is not. It is definitely not the network. The only real wildcards here are if, perhaps, this is trying to impersonate a user instead of running as the service account. The former would fail across the board, the latter should succeed. Then I wonder if the cert referenced in the config needs to be added to the DB server, or if Kerberos factors into some part of the connection...etc.
The config is how the installer set it up--no more, no less. If there are additional steps that need to be taken to set this up, I am completely unaware of them, but more than happy to perform them.
I'll be watching this and some other threads closely in the meantime and will add to this if I see this or we resolve it elsewhere.
I would be glad to help find out what that "something" is, to the extent that I can.
I'm using NTLM but also on a management server for my lab environment.
@john_doyle
have you seen similar issues in the configuration above?
For example, on the "Asset Analysis" dashboard page:
When run without SSL the New Hardware Asset view worked like a charm, it would actually load the 17k Hardware Assets in some seconds. (instead of minutes)
Then I remembered I didn't do any SSL settings in the initial Cireson Setup, I actually made the SSL settings by hand and everything seemed to work, except ODATA calls.
So I figured I would reinstall with selecting the Advanced Web Settings in the setup and this worked like a charm.
It might not solve all occurrences of this error but it might solve some.
What exactly could be going wrong in the JSON file that I should be looking at? I still have not been contacted by anyone since the day my ticket was sent to development.
It seems to matter very much whether your environment is configured to allow connections to /platform/api, BEFORE the installer runs. In other words, installing the portal then setting up your server to conform to all this endpoint's demands later does not seem to work.
I had a dev environment and a staging environment, both on portal v8.2.2, both with identical IIS configs. Dev worked, staging did not. That seems to be because we added an SPN and configured DTC in dev first, then rolled those changes to staging later, which happened to be after the portal was upgraded in both environments. I just now updated both environments to portal 8.3.1 and it fixed the issue in staging, while dev continues to work properly.
The DTC config I am referring to was mentioned by @John_Long in another thread (https://community.cireson.com/discussion/comment/10723#Comment_10723 ).
DTC hasn’t been required since v8 though so that shouldn’t be necessary.