The code is not using the correct winapi function. FindWindowEx() can find child windows, but the window displayed by MsgBox() is not a child window. It is a top-level window, the kind you could find with FindWindow().
But that function isn't good enough to locate the specific message box you want to close. A better approach is needed, one you could use is enumerating the windows that are owned by the same thread with EnumThreadWindows(). Nice thing about MsgBox() is that there will be only ever one such window since the dialog box is modal.
SendKeys() is not exacty good enough either, it only ever works properly if the message box is in the foreground. A much better approach is to actually click the button by sending it the BM_CLICK message. Tested code, using an Access form:
Imports System.Runtime.InteropServices
Imports System.ComponentModel
Imports System.Text
Module Module1
Sub Main()
'' Find the MS-Access host window
Dim access = FindWindow("OMain", Nothing)
If access = IntPtr.Zero Then Throw New Win32Exception()
'' Enumerate the windows owned by the same thread
Dim pid As Integer
Dim tid = GetWindowThreadProcessId(access, pid)
If tid = 0 Then Throw New Win32Exception()
EnumThreadWindows(tid, AddressOf ClickOkButton, Nothing)
End Sub
Private Function ClickOkButton(hWnd As IntPtr, lp As IntPtr) As Boolean
'' Verify the class name is #32770
Dim buf As New StringBuilder(256)
GetClassName(hWnd, buf, 256)
If buf.ToString <> "#32770" Then Return True
'' Find the OK button (control ID 2)
Dim okbutton = GetDlgItem(hWnd, 2)
If okbutton = IntPtr.Zero Then Return True
'' Activate the dialog, just in case
SetActiveWindow(hWnd)
'' Click the button
SendMessage(okbutton, BM_CLICK, IntPtr.Zero, IntPtr.Zero)
'' Done, no need to continue enumerating windows
Return False
End Function
End Module
Friend Module NativeMethods
<DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
Friend Function FindWindow(ByVal lpClassName As String, ByVal lpWindowName As String) As IntPtr
End Function
<DllImport("user32.dll", SetLastError:=True)>
Friend Function GetWindowThreadProcessId(ByVal hwnd As IntPtr, ByRef lpdwProcessId As Integer) As Integer
End Function
Friend Delegate Function EnumThreadDelegate(hWnd As IntPtr, lParam As IntPtr) As Boolean
<DllImport("user32.dll", SetLastError:=True)>
Friend Function EnumThreadWindows(dwThreadId As Int32, lpfn As EnumThreadDelegate, lParam As IntPtr) As Boolean
End Function
<DllImport("user32.dll", CharSet:=CharSet.Auto)>
Friend Function GetClassName(ByVal hWnd As System.IntPtr, ByVal lpClassName As System.Text.StringBuilder, ByVal nMaxCount As Integer) As Integer
End Function
<DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
Friend Function GetDlgItem(ByVal hDlg As IntPtr, id As Integer) As IntPtr
End Function
<DllImport("user32.dll", SetLastError:=True)>
Friend Function SetActiveWindow(ByVal hWnd As IntPtr) As IntPtr
End Function
<DllImport("user32.dll")>
Friend Function SendMessage(ByVal hWnd As IntPtr, ByVal msg As Integer, ByVal wp As IntPtr, ByVal lp As IntPtr) As IntPtr
End Function
Friend Const BM_CLICK As Integer = &HF5
End Module
Usual advice is to favor UI Automation.