Round corner for BottomSheetDialogFragment

create a shape named rounded_corners_shape

<shape xmlns:android="http://schemas.android.com/apk/res/android"
       android:shape="rectangle">
    <corners
        android:topLeftRadius="8dp"
        android:topRightRadius="8dp"/>
    <solid android:color="@color/white"/>

</shape>

define a style

  <style name="AppBottomSheetDialogTheme"
           parent="Theme.Design.Light.BottomSheetDialog">
        <item name="bottomSheetStyle">@style/AppModalStyle</item>
    </style>

    <style name="AppModalStyle" parent="Widget.Design.BottomSheet.Modal">
        <item name="android:background">@drawable/rounded_corners_shape</item>
    </style>

use this style on your custom BottomSheetDialogFragment like this, it will be work!

 public class CustomDialogFragment extends BottomSheetDialogFragment {
      @Override
      public void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setStyle(STYLE_NORMAL, R.style. AppBottomSheetDialogTheme);
      }

      ...
    }

The BottomSheetDialog is setting a default white background color , this is why the corners are not visible, In order to show them you need to make the background of the dialog transparent by overriding the style of the BottomSheetDialog.

Define this style In your res/values/styles/styles.xml

<style name="BottomSheetDialog" parent="Theme.Design.Light.BottomSheetDialog">
    <item name="bottomSheetStyle">@style/bottomSheetStyleWrapper</item>
</style>

<style name="bottomSheetStyleWrapper" parent="Widget.Design.BottomSheet.Modal">
    <item name="android:background">@android:color/transparent</item>
</style>

And set this style to your BottomSheetDialog

View view = getLayoutInflater().inflate(R.layout.chooser_bottom_sheet, null);
BottomSheetDialog dialog = new BottomSheetDialog(this,R.style.BottomSheetDialog); // Style here
dialog.setContentView(view);
dialog.show();

With the new Material Component library you can customize the shape of your component using the shapeAppearanceOverlay attribute in your style (Note: it requires at least the version 1.1.0)

Just use the BottomSheetDialogFragment overriding the onCreateView method and then define your custom style for Bottom Sheet Dialogs.

Define the bottomSheetDialogTheme attribute in styles.xml in your app theme:

  <!-- Base application theme. -->
  <style name="AppTheme" parent="Theme.MaterialComponents.Light">
    <!-- Customize your theme here. -->
    <item name="colorPrimary">@color/colorPrimary</item>
    ....
    <item name="bottomSheetDialogTheme">@style/CustomBottomSheetDialog</item>
  </style>

Then just define your favorite shape with shapeAppearanceOverlay

  <style name="CustomBottomSheetDialog" parent="@style/ThemeOverlay.MaterialComponents.BottomSheetDialog">
    <item name="bottomSheetStyle">@style/CustomBottomSheet</item>
  </style>

  <style name="CustomBottomSheet" parent="Widget.MaterialComponents.BottomSheet">
    <item name="shapeAppearanceOverlay">@style/CustomShapeAppearanceBottomSheetDialog</item>
  </style>

  <style name="CustomShapeAppearanceBottomSheetDialog" parent="">
    <item name="cornerFamily">rounded</item>
    <item name="cornerSizeTopRight">16dp</item>
    <item name="cornerSizeTopLeft">16dp</item>
    <item name="cornerSizeBottomRight">0dp</item>
    <item name="cornerSizeBottomLeft">0dp</item>
  </style>

---

You can obtain the same behavior overriding this method in your BottomSheetDialogFragment (instead of adding the bottomSheetDialogTheme in your app theme):

@Override public int getTheme() {
    return R.style.CustomBottomSheetDialog;
  }

In this case you are using this themeOverlay only in the single BottomSheetDialogFragment and not in all the app.

---

Important note about the EXPANDED STATE:

In the expanded state the BottomSheet has flat corners . You can check the official comment in github repo:

Our design team is strongly opinionated that rounded corners indicate scrollable content while flat corners indicate that there is no additional content. As such, they do no want us to add this change with fitToContents.

This behavior is provided by the BottomSheetBehavior and it is impossible to override it.
However there is a workaround -> DISCLAIMER: it can stop to work in the next releases !!

You can add a BottomSheetCallback in the BottomSheetDialogFragment:

  @NonNull @Override public Dialog onCreateDialog(@Nullable Bundle savedInstanceState) {
    Dialog dialog = super.onCreateDialog(savedInstanceState);


    ((BottomSheetDialog)dialog).getBehavior().addBottomSheetCallback(new BottomSheetBehavior.BottomSheetCallback() {

      @Override public void onStateChanged(@NonNull View bottomSheet, int newState) {
        if (newState == BottomSheetBehavior.STATE_EXPANDED) {
          //In the EXPANDED STATE apply a new MaterialShapeDrawable with rounded cornes
          MaterialShapeDrawable newMaterialShapeDrawable = createMaterialShapeDrawable(bottomSheet);
          ViewCompat.setBackground(bottomSheet, newMaterialShapeDrawable);
        }
      }

      @Override public void onSlide(@NonNull View bottomSheet, float slideOffset) {

      }
    });

    return dialog;
  }

  @NotNull private MaterialShapeDrawable createMaterialShapeDrawable(@NonNull View bottomSheet) {
    ShapeAppearanceModel shapeAppearanceModel =

      //Create a ShapeAppearanceModel with the same shapeAppearanceOverlay used in the style
      ShapeAppearanceModel.builder(getContext(), 0, R.style.CustomShapeAppearanceBottomSheetDialog)
        .build();

      //Create a new MaterialShapeDrawable (you can't use the original MaterialShapeDrawable in the BottoSheet)
      MaterialShapeDrawable currentMaterialShapeDrawable = (MaterialShapeDrawable) bottomSheet.getBackground();
      MaterialShapeDrawable newMaterialShapeDrawable = new MaterialShapeDrawable((shapeAppearanceModel));
      //Copy the attributes in the new MaterialShapeDrawable
      newMaterialShapeDrawable.initializeElevationOverlay(getContext());
      newMaterialShapeDrawable.setFillColor(currentMaterialShapeDrawable.getFillColor());
      newMaterialShapeDrawable.setTintList(currentMaterialShapeDrawable.getTintList());
      newMaterialShapeDrawable.setElevation(currentMaterialShapeDrawable.getElevation());
      newMaterialShapeDrawable.setStrokeWidth(currentMaterialShapeDrawable.getStrokeWidth());
      newMaterialShapeDrawable.setStrokeColor(currentMaterialShapeDrawable.getStrokeColor());
      return newMaterialShapeDrawable;
  }

Create a custom drawable rounded_dialog.xml:

<?xml version="1.0" encoding="utf-8"?>
<shape xmlns:android="http://schemas.android.com/apk/res/android"
    android:shape="rectangle">
    <solid android:color="@android:color/white"/>
    <corners android:topLeftRadius="16dp"
        android:topRightRadius="16dp"/>

</shape>

Then override bottomSheetDialogTheme on styles.xml using the drawable as background:

<style name="AppTheme" parent="Theme.AppCompat.Light.NoActionBar">       
    <item name="bottomSheetDialogTheme">@style/AppBottomSheetDialogTheme</item>
</style>

<style name="AppBottomSheetDialogTheme"
    parent="Theme.Design.Light.BottomSheetDialog">
    <item name="bottomSheetStyle">@style/AppModalStyle</item>
</style>

<style name="AppModalStyle"
    parent="Widget.Design.BottomSheet.Modal">
    <item name="android:background">@drawable/rounded_dialog</item>
</style>

This will change all the BottomSheetDialogs of your app.