How to remove border from Dialog?

The border, round corners and margin are all defined by android:windowBackground. (Parameter android:windowFrame is already set to @null in Theme.Dialog style, therefore setting it to @null again has no effect.)

In order to remove the border and round corners you have to change the android:windowBackground appropriately. The Theme.Dialog style sets it to @android:drawable/panel_background. Which is a 9-patch drawable that looks like this (this one is the hdpi version):

As you can see the 9-patch png defines the margin, border and round corners of the dialog theme. To remove the border and round corners you have to create an appropriate drawable. If you want to keep the shadow gradient you have to create set of new 9-patch drawables (one drawable for each dpi). If you don't need the shadow gradient you can create a shape drawable.

The required style is then:

<style name="ActivityDialog" parent="@android:style/Theme.Dialog">      
   <item name="android:windowBackground">@drawable/my_custom_dialog_background</item>            
</style>

Without creating a custom background drawable and adding a special style just add one line to your code:

dialog.getWindow().setBackgroundDrawableResource(android.R.color.transparent);

I played around a bit with other possibilities but using a 9 patch with fixed margins and found out that the layer-list drawable is allowing to define offsets, hence margins around its enclosed drawables, so this worked for me:

<?xml version="1.0" encoding="utf-8"?>
<layer-list xmlns:android="http://schemas.android.com/apk/res/android" >

    <item
        android:drawable="@drawable/my_custom_background"
        android:top="5dp" android:bottom="5dp" android:left="5dp" android:right="5dp">
    </item>

</layer-list>

and then you can use this as the "android:windowBackground":

<style name="ActivityDialog" parent="@android:style/Theme.Dialog">      
   <item name="android:windowBackground">@drawable/my_custom_layer_background</item>            
</style>