Let's start with Google recommendation which says to use DialogFragment instead of a simple Dialog.
@rekire is right that margins set by drawable, going forward it is set by either 9 patch or programmatically depending on theme.
So you either can set your padding to your content view or create dialog using DialogFragment here is an example which changes height of dialog based on it's content, and note you don't need to use tree observer which is as mentioned before may cause performance issue.
So the example
dialog_confirm.xml
<?xml version="1.0" encoding="utf-8"?>
<ScrollView xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:padding="20dp">
<LinearLayout android:id="@+id/container"
xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:background="@android:color/white"
android:orientation="vertical"
android:animateLayoutChanges="true"
android:padding="15dp">
<TextView
android:id="@+id/textView"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center_horizontal"
android:padding="10dp"
android:text="A label text"
android:textAppearance="?android:attr/textAppearanceLarge"/>
<TextView
android:id="@+id/textView2"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center_horizontal"
android:padding="10dp"
android:text="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque mauris mi, dictum a lectus ut, facilisis"
android:textAppearance="?android:attr/textAppearanceMedium"/>
<Button
android:id="@+id/button1"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center_horizontal"
android:text="Remove Me"/>
<Button
android:id="@+id/button2"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center_horizontal"
android:text="Remove Me"/>
<Button
android:id="@+id/button3"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center_horizontal"
android:text="Remove Me"/>
<!-- as much content as you need -->
</LinearLayout>
</ScrollView>
Note: I wrapped everything into scroll view and set padding you can skip it if you want.
ConfirmDialog.java
//here goes package name and imports
/**
* Created by Vilen - virtoos.com;
* fragment dialog example
*/
public class ConfirmDialog extends DialogFragment implements View.OnClickListener {
private Button button1;
private Button button2;
private Button button3;
private LinearLayout containerLayout;
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setStyle(DialogFragment.STYLE_NO_TITLE, 0);
}
@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container,
Bundle savedInstanceState) {
View v = inflater.inflate(R.layout.dialog_confirm, container, false);
containerLayout = (LinearLayout)v.findViewById(R.id.container);
button1 = (Button)v.findViewById(R.id.button1);
button2 = (Button)v.findViewById(R.id.button2);
button3 = (Button)v.findViewById(R.id.button3);
button1.setOnClickListener(this);
button2.setOnClickListener(this);
button3.setOnClickListener(this);
return v;
}
@Override
public void onActivityCreated(Bundle savedInstanceState) {
super.onActivityCreated(savedInstanceState);
// make background transparent if you want
//getDialog().getWindow().setBackgroundDrawable(new ColorDrawable(Color.TRANSPARENT));
}
@NonNull
@Override
public Dialog onCreateDialog(Bundle savedInstanceState) {
return super.onCreateDialog(savedInstanceState);
}
@Override
public void onClick(View v) {
switch (v.getId()){
case R.id.button1:
containerLayout.removeView(button1);
break;
case R.id.button2:
containerLayout.removeView(button2);
break;
case R.id.button3:
containerLayout.removeView(button3);
break;
}
}
}
and finally you can show your dialog with this piece of code
ConfirmDialog confirmDialog = new ConfirmDialog();
confirmDialog.show(getSupportFragmentManager(), "dialog");

I will not go into details why Fragment dialog is better but one thing is clear that you can encapsulate logic for it and have separate class.
Hope this solves your issue.