How to specify Marshalling for the function MAPISendMail

Question

Since upgrading to Visual Studio 16.8.1, I am getting many errors from the code analysis in my (somewhat old) VB.NET project.

One error is on the following DLLImport

<DllImport("MAPI32.DLL")> _
Private Shared Function MAPISendMail ( ByVal sess     As IntPtr, _
                                       ByVal hwnd     As IntPtr, _
                                       ByVal message  As MapiMessage, _
                                       ByVal flg      As Integer, _
                                       ByVal rsv      As Integer ) As Integer
End Function

The error is

Error CA2101
To reduce security risk, marshal field 'MapiMessage.subject' as Unicode, by setting StructLayout.CharSet on 'MapiMessage' to CharSet.Unicode, or by explicitly marshaling the field as UnmanagedType.LPWStr. If you need to marshal this string as ANSI or system-dependent, specify MarshalAs explicitly, use the BestFitMapping attribute to turn best-fit mapping off, and for added security, to turn ThrowOnUnmappableChar on.

The structure MAPI message is defined as

<StructLayout(LayoutKind.Sequential)> _
Public Class MapiMessage
  Public reserved       As Integer
  Public subject        As String
  Public noteText       As String
  Public messageType    As String
  Public dateReceived   As String
  Public conversationID As String
  Public flags          As Integer
  Public originator     As IntPtr
  Public recipCount     As Integer
  Public recips         As IntPtr
  Public fileCount      As Integer
  Public files          As IntPtr
End Class

I have tried specifying CharSet.Ansi on both the function

<DllImport("MAPI32.DLL", CharSet := CharSet.Ansi)> _
Private Shared Function MAPISendMail ( ByVal sess     As IntPtr, _
                                       ByVal hwnd     As IntPtr, _
                                       ByVal message  As MapiMessage, _
                                       ByVal flg      As Integer, _
                                       ByVal rsv      As Integer ) As Integer
End Function

and the structure

<StructLayout(LayoutKind.Sequential, CharSet := CharSet.Ansi)> _
Public Class MapiMessage
  ...
End Class

and also tried specifying MarshalAs(UnmanagedType.LPStr) on the field subject.

<MarshalAs(UnmanagedType.LPStr)> _
Public subject        As String

I have also tried specifying BestFitMapping = false and ThrowOnUnmappableChar = true, as suggested in the error message.

<DllImport("MAPI32.DLL", CharSet := CharSet.Ansi, BestFitMapping := False, ThrowOnUnmappableChar := True)> _
Private Shared Function MAPISendMail ( ByVal sess     As IntPtr, _
                                       ByVal hwnd     As IntPtr, _
                                       ByVal message  As MapiMessage, _
                                       ByVal flg      As Integer, _
                                       ByVal rsv      As Integer ) As Integer
End Function

I am still getting the same error.

I have not yet tried using MAPISendMailW (and I haven't found a sample), but even if that provides a solution, surely there must be a way to marshal as ANSI without an error message.

What exactly do I have to do to fix this error?

See following : http://www.pinvoke.net/default.aspx/mapi32.MAPISendMail — jdweng, Nov 14 '20 at 13:40
There is a difference how string are handled in Basic and c++. A Basic string the length is at the beginning of the object while in c++ string are terminated with a '\0'. See following : https://learn.microsoft.com/en-us/dotnet/framework/interop/default-marshaling-for-strings — jdweng, Nov 14 '20 at 13:45
@jdweng My code pretty much corresponds to the example in that link. I'm pretty sure that if I cut and pasted the code in that link into a new project, in VB or C#, I would get exactly the same error message. My problem is not with the code itself, but with the error CA2101. I would like to define attributes correctly, so that the error is not generated. — Phil Jollans, Nov 14 '20 at 14:18
According to my understanding, would be correct for a C++ string, with null termination with the ANSI character set. — Phil Jollans, Nov 14 '20 at 14:19
You may not like this, but you should read: [FYI: Why are MAPI and CDO 1.21 not supported in managed (.NET) code?](https://learn.microsoft.com/en-us/archive/blogs/mstehle/fyi-why-are-mapi-and-cdo-1-21-not-supported-in-managed-net-code) — TnTinMn, Nov 14 '20 at 15:44
No problem, I'm sure this is not really good code. It is also very old and should probably be rewritten. But until I do, I would like to get rid of the error message. My solution today will almost certainly be to disable the code analysis, which is probably the worst possible solution. — Phil Jollans, Nov 14 '20 at 16:12
Looks like you can still use code. See : https://learn.microsoft.com/en-us/windows/win32/api/mapi/nc-mapi-mapisendmail — jdweng, Nov 14 '20 at 22:52
@jdweng Thanks, I will look into that. I know that I have an outstanding problem sending mail when the user's mail program is Outlook (although other mail programs work fine), so I should fix the code anyway. — Phil Jollans, Nov 15 '20 at 07:27
Getting code to work depends on the project settings. I suspect you code will work with x86 mode and 32 bit mode but you will need to use MAPISendMailW for 64 bit mode. — jdweng, Nov 15 '20 at 09:09

score 1 · Accepted Answer · answered Nov 15 '20 at 00:15

The only way I could get the code analysis error you are experiencing was by enabling the "Binary analyzers - Run on build option" that has been depreciated.

The following combination of attributes eliminated the "CA2101" errors.

<DllImport("MAPI32.DLL")>
Private Shared Function MAPISendMail(ByVal sess As IntPtr,
                                      ByVal hwnd As IntPtr,
                                      ByVal message As MapiMessage,
                                      ByVal flg As Integer,
                                      ByVal rsv As Integer) As Integer
End Function

<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Ansi),
              BestFitMapping(False, ThrowOnUnmappableChar:=True)>
Public Class MapiMessage
  Public reserved As Integer
  <MarshalAs(UnmanagedType.LPTStr)> Public subject As String
  <MarshalAs(UnmanagedType.LPTStr)> Public noteText As String
  <MarshalAs(UnmanagedType.LPTStr)> Public messageType As String
  <MarshalAs(UnmanagedType.LPTStr)> Public dateReceived As String
  <MarshalAs(UnmanagedType.LPTStr)> Public conversationID As String
  Public flags As Integer
  Public originator As IntPtr
  Public recipCount As Integer
  Public recips As IntPtr
  Public fileCount As Integer
  Public files As IntPtr
End Class

Thanks, that works. What I had missed was how to apply the BestFitMapping attribute to the structure definition. — Phil Jollans, Nov 15 '20 at 07:29
I had seen that the binary analyzers are deprecated and I will update my project, but something has changed with the update to Visual Studio 16.8.1. It appears to be ignoring the settings in my ruleset file (and also my .editorconfig file), and marking everything as an error, even if the ruleset specifies that it should be a warning. — Phil Jollans, Nov 15 '20 at 07:36

How to specify Marshalling for the function MAPISendMail

1 Answers1