DOSBox (current version 0.74) has VESA functionality, provided the options file has the required settings. Some of the settings in my options file are:
[sdl]
fullscreen=false
fulldouble=false
fullresolution=desktop
windowresolution=1280x960
output=openglnb
[dosbox]
machine=svga_s3
[render]
aspect=true
scaler=normal2x
I wrote this little demo to prove that it works:

org 256
mov bx, 4115h ; 32-bit 800x600 graphics
mov ax, 4F02h ; VESA.SetVideoMode
int 10h ; -> AX
cmp ax, 004Fh
jne Abort
call DrawDiagonals
mov ah, 00h ; BIOS.GetKeystroke
int 16h ; -> AX
mov ax, 0003h ; BIOS.SetVideoMode 80x25 text
int 10h
Abort:
mov ax, 4C00h ; DOS.Terminate
int 21h
; ----------------------
DrawDiagonals:
mov cx, 600
xor eax, eax
mov ebx, 3200-4
.a:
mov dword [0C0000000h+eax], 004080C0h ; 0RGB
mov dword [0C0000000h+ebx], 00C08040h ; 0RGB
add eax, 3200+4
add ebx, 3200-4
loop .a
ret
; ----------------------
In a full-fledged application you would first ask VESA for its list with supported video modes, and then pick one for which you retrieve the characteristics (like LinearFrameBuffer address, BytesPerScanline, ...).
Consult the VESA documentation.