An SMlets based Exchange Connector

Tom_Hendricks

Using a forwarding rule in SCSM_DEV_Desktop instead of a redirection rule seems like the most likely culprit.  This feature is based on the email in your SCSM_DEV mailbox still appearing to have been sent to SCSM_DEV_Desktop.  Redirection preserves that, forwarding does not.

Next, it could potentially be due to BCCing it instead of having the address out in the open (EWS will not send back the BCC addresses, Resent-From, or other header info that Exchange has on the emails but does not wish to share).  If it is sent to both mailboxes, it should use whatever it finds in the $Mailboxes hashtable and ignore the default.

You might also want to double-check the IRTemplate name to make sure it is an exact match.  There is a lot of punctuation there, so perhaps there is a double-space (or needs to be), etc.?  If the template specified here is not found, I believe it will use the default.

It could be something else, but hopefully these ideas set you in the right direction!

Alan_Foster

Thanks @Tom_Hendricks
I think that my issue is that it was just forwarded and not actually redirected.  I spoke with the exchange admin briefly Friday afternoon but he was focused on another more pressing patching issue.  I hope to get this resolved this morning.

Alan_Foster

@Tom_Hendricks the email when it gets to SCSM_Dev does indicate that the TO: is SCSM_DEV_Desktop but it still does not use the correct template.  I have retraced the template names and copied them directly from the console.

$defaultNewWorkItem = "ir"

$defaultIRTemplateName = "NES - Default Incident Template"

...

$UseMailboxRedirection = $false

$Mailboxes = @{

    "SCSM_DEV_Desktop@nespower.com" = @{"DefaultWiType"="IR";"IRTemplate"="NES - Incident - Auto - Desktop Alert"};

}

Adam_Dzyacky

Catching up on this - don't forget to set $usemailboxredirection = $true if the above was a direct copy from your connector.

Alan_Foster

So, I went back to scratch.  Downloaded the files clean. Went through each setting and typed it all again instead of copy/paste.  @Adam_Dzyacky thanks for the catch, i had not set the $usemailboxredirection =  $true on the last run through.  I must have had something a miss in the previous ps1 file and or the exchange admin fixed something.  it is working now.  Thanks for all your help.  This Cireson Community is the best support for any software I have ever used. @Tom_Hendricks

Alan_Foster

@Adam_Dzyacky , @Tom_Hendricks
Well it worked once, but now it does not pick the right template.

Is there a log file created anywhere?

Gerhard_Goossens

This looks really awesome. So here is my strange question...
Will I be able to use this to use with GroupWise email? 
We do not use exchange and my knowledge regarding Mail, SMTP, IMAT etc is extremely limited.
I will try and read the code but thought I will ask here before I start digging.

Regards
Gerhard

Roland_Kind

@Gerhard_Goossens

I think it depends only how a Groupwise Mailbox can be accessed from powershell.

Maybe there is a SOAP/WebService Interface to Groupwise mailboxes - but of course, this needs some adjustments to the smlets powershell script

regards

Roland

Gerhard_Goossens

@Roland_Kind we do have a SOAP/WebMail service running, I will see what I can do. 
Thanks

Adam_Dzyacky

If you get it going, I'd be very interested to see the extent of the modifications required @Gerhard_Goossens. Especially if they could merged with the main repo on Github. If you're looking for the "core" of the connector and where you could start editing I have a blogpost on it here.

@Alan_Foster - There currently isn't but it has been suggested. Testing, troubleshooting, and QA-ing happens with breakpointing those areas in question and running the script as the workflow account/Service Manager email. All of this said, I'll try and set some time aside in the next week or so to start writing up documentation on the specifics of this feature from an Exchange perspective on the Wiki.

Gerhard_Goossens

@Adam_Dzyacky I completely forgot that we have an Office 365 subscription. Im busy setting up an account now. Will post progress...

Gerhard_Goossens

Ok, so I got it going, the script is connecting to the 365 exchange, pulling the mail and deleting it.
The problem I have is that our email is sent as name.surname@domain1.com and the AD accounts inside SCSM is 123456@domain2.com. This causes the script to not add the email comment, it does, however, add the email as an attachment created by name.surname@domain1.com. It also creates a new SCSM user name.surname@domain1.com. I have turned off the option to create a new user, so that is not an issue anymore.

I guess I can change the staff email address inside SCSM to reflect the name.surname email but for students this will not be possible because we provision them with Gmail addresses or they can use their own email provider.

I was thinking of changing the script so that it searches for the relationship that the email belongs to and then adding that user as the affected user, we have 60k users so that might take a while. And there might be an issue if the mail is sent from an email address that does not belong to a user ie. user changed his email address and it has not synced through yet.

I suspect that I will have the same issue if I use the stock SCSM exchange connector.

Another problem I have is that we will be using some service email accounts to send the email and thus the affected user will always be the wrong user. I was thinking of using SCORCH to grab the newly created IR or SR and searching inside the description for the affected user ie. 12345678 and then changing the relationship of the affected user. This approach will add more action logs though.

This all seems like an uphill battle for me and the lack of standards ie. multiple domains and different email providers on our side is killing me. 

NazOsman

I am not sure where to go for help, please direct me as you see fit.

If the request is a Service Request and the end user responds back to an email sent to them, the reply never gets logged as an "End User Comment". We have been missing all replies back from users when they are Service Requests.

Incident Requests seems to function just fine.

Please help,

Naz

Adam_Dzyacky

Raised - will investigate.

Adam_Dzyacky

Thank you @NazOsman. This will be addressed in an emergency Pull Request.

Conner_Wood

Wow, this is more advanced than I thought after reading through it all on GitHub.  This is what has been needed years ago.

Adam_Dzyacky

It's that time again!
1.4.5 has arrived on GitHub:

Azure Cognitive Services, Priority Scoring: This goes back to an idea I had awhile ago then recently raised by @NazOsman. If you're using ACS to dynamically create an Incident or Service Request based on some minimum sentiment score then why not further leverage this score to define the Impact, Urgency, and Priority on said New Work Item? Now you can define custom integer ranges wherein these enums will be set based on the Affected User's perceived sentiment returned from ACS.

Suggestion Email's Suggestions Sort by Relevance: Instead of grabbing Knowledge Articles and/or Request Offerings and then just sending them out to the Affected User. They will now sort based on words matched before going out.

Custom Suggestion Email Templates: Up until this point, if you enabled Suggest RO/KA from the Cireson SCSM Portal the email was hardcoded into the connector and required editing every single upgrade. Now they've been set free in their own distinct files enabling you to customize these HTML based templates to your organization's content.

Redact Sensitive Information: For the security conscience amongst us, we can't control what Affected Users will send in an email. But now we can control what actually makes it into SCSM, Cireson Analytics, and the DW by leveraging an optional regex text file that will scan emails before they are committed into SCSM replacing sensitive information with [redacted]. Thanks go out to nradler2 for wholly executing on this feature request!

While a minor update, perhaps a feature or two here is something you're interested in but you aren't sure the best way to upgrade your connector short of writing down and resetting various $true/$false values? No worries because the SMLets Exchange Connector blog has got you covered showing you how to quickly spot check between your customized connector and the latest version. Maybe you have an idea how to make the connector more awesome? Submit a Feature Request with your GitHub account today!

Kenneth_McMichael

This is great. I have been testing it out and can't seem to find the stock setting that does this


Creating the Incident initially seems to be okay but I can't find where to apply a template when it's updated.

Adam_Dzyacky

It's only as I'm typing this and staring down your screenshot that I can now plainly see the use case/point being made here @Kenneth_McMichael given your choice of Template Title.

That said - you're correct in your observation. There isn't an update template in the connector. However if the only end game here is to set the Incident's Status property to Active (again, I'm assuming only going off of the Title of your Template) that can be achieved rather easily with a single addition to the connector. But ultimately, probably something worth adding. If you have a GitHub account can you raise this request on the repo?

Kenneth_McMichael

Thanks for the fast response! Your assumption is correct - it is being used to change it to Active (from pending status). I have raised this request on github and starred!

Kenneth_McMichael

I ran into an issue using the default template for an IR but was able to resolve myself. I didn't find anywhere that some had this exact problem so posting my findings below

I configured this value here

$defaultIRTemplateName = "Default Incident Template"

But it would not use that template and if I tried others I had the same result it would default to the new template settings.

After tracing the ps command to Get-SCSMObjectTemplate -DisplayName $DefaultIRTemplateName

I ran this command by itself and was getting multiple names returned.

I had a 'Default Incident Template' and 'Default Incident Template - Portal' - it didn't know which one to choose and was failing.

As a quick fix I created a template with a unique name that the command above would only return a single result and this fixed the issue.

Tom_Hendricks

The SMlets like to treat strings as a search rather than an exact sting match.  Adding a dollar sign to the end of your template name would prevent matches that have more characters, though.  (You can see this elsewhere throughout the script such as the block where all the classes are loaded and you'll see "System.WorkItem$" to prevent matching "System.WorkItem.Incident" for example).

Using your example:

$defaultIRTemplateName = "Default Incident Template$"

It looks like you have handled this just fine, but I want to comment in case anyone else has an issue with this, and also because a brief comment about this in the script configuration block seems like a really good idea now, in addition to modifying the code to only take the first result instead of assuming there will be only one.

Adam_Dzyacky

I was just about to suggest piping this into a Where-Object

Get-SCSMObjectTemplate -DisplayName "$defaultIRTemplateName" | where-object ?{$_.displayname -eq "$defaultIRTemplateName"}

To GitHub @Tom_Hendricks !

Adrian_Paech

Hey Adam,
Love your work with this addon.
To work in our environment, I had to make a few changes, but nothing too major.
However, some of the following adjustments I made could be added into future versions if you feel they are worthwhile.
These changes included:

1. Being able to disable the "auto creating workitems" feature based on a template when there is no associated work item ID found, or no work item ID specified. We did not want this feature switched on, as we do not want jobs being automatically created via email. (as is the case with your code and the default exchange connector). For us we require our Help Desk to intervene and review the email before it is logged. 
2. If disabling the "auto creating workitems" feature (as described above), specify an alternate email address where these emails without work item ID's will be forwarded to (such as the Help Desk Inbox), so that non-processed emails can still be dealt with by the Service Desk via the standard email request process and don't get missed.
3. Instead of only putting the body of the email in the notes field, include the "From, To, CC, Subject and Date Sent" fields in the notes also. This prevents us from having to attach the emails separately, which provides a better audit trail, and saves on DB storage requirements.
4. Email Error reporting.. We can specify an administrator email address (or log file) so that admins can be alerted of when code errors occur, what the error was, what the line of code was (+line number) etc. and can address these issues before getting complaints from the Help Desk or users.

Further, 
I did find the code a little confusing to follow. 
Although I didn't have time to do this myself, it might be an idea to make the code slightly more modular so that its a little easier to read and modify etc. Especially considering how many lines of code there are. 

Hope this helps.
Adrian

Adam_Dzyacky

Just the kind of news and insight I enjoy getting @Adrian_Paech! I'll preface the following with this - feel free to PM if you want as I sense we could have some lengthy, but truly productive back n forth on these points and in the interests of everyone's inbox notifications it may make more sense. With that said...

1 & 2: Certainly sounds like an organizational specific process. In which case I'm not sure (at least off the top of my head) how this could be addressed for everyone using the smlets exchange connector given this is the crux of what it and the stock Microsoft one does.

3: What if this was the default action when you aren't attaching emails to Work Items? I'd be curious to know from others using this if there was a larger need to do this regardless of the feature being enabled or not.

4: You're certainly not the first to bring up a logging feature, but you are the first to bring up this way of doing it. I must admit there has been a lack of action here on my part as I don't know where people will take this and end up running the connector. I originally wanted to do logging in the Windows Event Log on the WF server but if you're running this in SMA the output per job also satisfies the request. Which means we're at about 3 different proposed ways to do logging. All valid. All great. All different. If this is something you can wire up and contribute, I'd be open to reviewing it to at least get some mechanism of logging in place. It seems like it could lead to a lot of overlap, but maybe it just needs to be a external logging script and everyone gets to pick how the logging is done between the various means of performing it on the connector? Lot of good and interesting discussion to be had here for sure.

In regards to following along through the code, myself and others have certainly tried to leverage functions where at all possible to keep this modular. I'd like to think we've made a lot of great progress since the first version released over a year ago but that said, there is always room for improvement. Whether it be more inline comments, more succinct ways of writing some PowerShell, or raising Issues on Github. I am certainly welcoming of any contributions you'd be willing to make on the repo in the name of cleaner code or less lines.

Tom_Hendricks

Just injecting some more opinions here (why not? ).  On 1 & 2 I agree that it is an organizational process and that certainly not everyone would want to run their workflow in that way.  However, I like thinking that this could be supported modularly.  I had to make up a word to make my point, so I'll use 2 points to clarify:

• The script should not get in the way of that type of customization
• The script should allow that type of customization, even if it does not contain it 

Supporting those two points might be trickier, but I think it can be done reasonably.  What first springs to mind is that perhaps more default behaviors currently in the script could be enabled/disabled (enabled by default, probably) and that the custom events script could be used to launch said customizations.

Regarding #3: I like your idea, @Adam_Dzyacky.  Doing nothing if email attachment to tickets is disabled does not seem like a popular choice.  Maybe a poll of some kind either here or on GitHub could settle that...?

#4/logging: SMA fills that requirement for me, so I do not have much of a stake in this point except for this--if any built-in logging is added, it should be possible to disable it.  As a technologist who has thoughts on it anyway, even though it doesn't pertain to me, writing to the Windows (is anyone running this elsewhere?) event log seems like the option with the broadest support and least configuration.  There are certainly "better" ways to log, but this is the one that would probably work for anyone who wants it, with minimal fuss.

Ben_McGarry

Can't get the SMLets Exchange Connector working (I have had it running before in a previous life and think it is "the business").  Would really appreciate some help with this as I want to ditch the Microsoft Exchange Connector with which we are experiencing some "anomalies".  When the smletsConnector script runs is produces the following error:

(line 376 in unedited/latest version of script)
$defaultIRTemplate = Get-SCSMObjectTemplate -DisplayName $DefaultIRTemplateName @scsmMGMTParams

cmd : Get-SCSMObjectTemplate : Value cannot be null.

At line:1 char:1

+ cmd /c C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -fil ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (Get-SCSMObjectT...cannot be null.:String) [], RemoteException

    + FullyQualifiedErrorId : NativeCommandError

 

Parameter name: input

At C:\smletsExchangeConnector\smletsExchangeConnector.ps1:384 char:26

+ ... RTemplate = Get-SCSMObjectTemplate -DisplayName $DefaultIRTemplateNam ...

+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Get-SCSMObjectTemplate], ArgumentNullException

    + FullyQualifiedErrorId : System.ArgumentNullException,SMLets.GetSCSMObjectTemplateCommand

NOTE:  I then changed the command to use -Name rather than -DisplayName (and hard-coded the template names) and got around the error but discovered the returned value is blank in $defaultIRTemplate:
$defaultIRTemplate = Get-SCSMObjectTemplate -Name "EmailDefaultIRTemplateName"  @scsmMGMTParams

The script now runs through with no errors but does not read the mail from the mailbox.

Any ideas?

Thanks a mil,
Ben

Adam_Dzyacky

@Ben_McGarry - the next release (1.4.6) addresses this potential issue with Get-SCSMObjectTemplate in the event you have similar template names in SCSM through some PowerShell piping.

As far as the script runs but does not read from the mailbox I'd suggest you try an incredibly trimmed down version of the connector I wrote whose sole purpose is to demonstrate connectivity by simply looping through the messages within a PowerShell window. It doesn't alter the inbox in anyway, so any subsequent run of the actual connector isn't destined for failure. You can grab it here on the SMLets Exchange Connector blog.

Adam_Dzyacky

Before the year is over, it's time for one last release of the SMLets Exchange Connector with v1.4.6!

AI without Azure Cognitive Services: This past year, ACS was introduced as an option to create an Incident or Service Request based on the Affected User's perceived attitude. In doing so, this curbed a long standing issue that the original connector exposed - you must create either Incidents or Service Requests by default. Regardless of what you choose, it inevitably leads to the conversation that Analysts most often bring up and that's "Why can't I convert Incidents to Service Requests and vice versa?" Recognizing that ACS may not be for everyone but the feature still warranted, @Tom_Hendricks has introduced an alternative to ACS that allows you to define a list of keywords (e.g. error, problem, fail, crash, etc.) that will trigger the creation of an Incident otherwise a Service Request will be created.

Multiple Mailbox Routing enhancements: @Tom_Hendricks continues to expand this feature out that now supports processing multiple mailboxes that are BCCed on the original email by using PowerShell to interrogate the email headers before the connector performs additional processing.

Change Incident Status on Update: Now you can now change the Status of Incidents when the Affected User, Assigned To, or Related User replies. This helps better reflect time within a specific status as Analysts flip to Pending and potential Pending subcategories. As an example, this means you can configure the connector so that the Assigned To updates it to Pending and the Affected User updates it to Active. Thus further complimenting the Cireson Send Email feature on the portal. Thanks to @Kenneth_McMichael for raising this process feature request!

general optimizations: This release was rather heavy on optimizations, so the feature list above isn't as large as previous updates. These optimizations include items such as introducing new functions, combining/simplifying others, leveraging others introduced on previous releases more and performing things like moving away from Invoke-WebRequest to Invoke-RestMethod. A full list of these optimizations can be seen on the 1.4.6 milestone here. 


The SMLets Exchange Connector has grown quite a bit this past year and even more so since its first release in April of 2017. All of it leading to an ever increasing adoption across SCSM deployments of all shapes and sizes. The latest adopter? It's one many of us here in the community know as I recently co-authored a blog post with @Joe_Burrows as he's deployed it for Cireson on Azure Automation for their SCSM deployment. You can read more about it here - Cireson Support Deploys Open Source SMLets Based Exchange Connector

Justin_Workman

This connector is great!  Awesome work here!  Congrats to @Adam_Dzyacky, @Tom_Hendricks, and everyone else who contributes!  I look forward to seeing the progress continue in 2019!