First of you shall understand that if you remove hidden modal View Controller (A), which presented your (B), then (B) also dismisses.
Since you asked to remove (A) from (B), then you need to pass parent of view controller, you wish to dismiss, in your case is (M) is parent for (A), so your (B), should know (M). Then, you do
(M).dismiss(animated: true, completion: nil)
which means view controller M will dismiss it's child view controller, which is (A) for your case.
The main point here is to call dismiss on parent view controller. Explanation could be found in description of dismiss provided by Apple (I highlighted parts, you should give your attention): -
*
Dismisses the view controller that was presented modally by the view
controller. The presenting view controller is responsible for
dismissing the view controller it presented. If you call this method
on the presented view controller itself, UIKit asks the presenting
view controller to handle the dismissal. If you present several view
controllers in succession, thus building a stack of presented view
controllers, calling this method on a view controller lower in the
stack dismisses its immediate child view controller and all view
controllers above that child on the stack. When this happens, only the
top-most view is dismissed in an animated fashion; any intermediate
view controllers are simply removed from the stack. The top-most view
is dismissed using its modal transition style, which may differ from
the styles used by other view controllers lower in the stack. If you
want to retain a reference to the view controller's presented view
controller, get the value in the presentedViewController property
before calling this method. The completion handler is called after the
viewDidDisappear(_:) method is called on the presented view
controller.
*