Register to be default app for custom file type

One possible answer is shown here . Use this inside one of your activity tag and it'll be opened when the file have been pressed. I tested it using the native android file manager in Oreo.

<intent-filter android:priority="999">
    <action android:name="android.intent.action.VIEW" />

    <category android:name="android.intent.category.DEFAULT" />
    <category android:name="android.intent.category.BROWSABLE" />
    <category android:name="android.intent.category.OPENABLE" />

    <data android:host="*" />
    <data android:mimeType="application/octet-stream" />
    <data android:pathPattern=".*\\..*\\..*\\..*\\..*\\.yourextension" />
    <data android:pathPattern=".*\\..*\\..*\\..*\\.yourextension" />
    <data android:pathPattern=".*\\..*\\..*\\.yourextension" />
    <data android:pathPattern=".*\\..*\\.yourextension" />
    <data android:pathPattern=".*\\.yourextension" />
    <data android:scheme="content" />
</intent-filter>

Here is a fuller example to show how to register your app to open plain text files.

Register the type of file you can handle

Add an intent-filter to your activity in the manifest. In my case it is the ReaderActivity that will display the text file. The mimeType says that I will accept any plain text file. See this for specific file extensions.

<activity android:name=".ReaderActivity">
    <intent-filter >
        <action android:name="android.intent.action.VIEW" />
        <category android:name="android.intent.category.DEFAULT" />
        <data android:scheme="file"/>
        <data android:mimeType="text/plain"/>
    </intent-filter>
</activity>

Handle the intent

Get the data from the intent in your activity like this:

public class ReaderActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_reader);

        handleIntent();
    }

    private void handleIntent() {

        Uri uri = getIntent().getData();
        if (uri == null) {
            tellUserThatCouldntOpenFile();
            return;
        }

        String text = null;
        try {
            InputStream inputStream = getContentResolver().openInputStream(uri);
            text = getStringFromInputStream(inputStream);
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

        if (text == null) {
            tellUserThatCouldntOpenFile();
            return;
        }

        TextView textView = findViewById(R.id.tv_content);
        textView.setText(text);
    }

    private void tellUserThatCouldntOpenFile() {
        Toast.makeText(this, getString(R.string.could_not_open_file), Toast.LENGTH_SHORT).show();
    }

    public static String getStringFromInputStream(InputStream stream) throws IOException {
        int n = 0;
        char[] buffer = new char[1024 * 4];
        InputStreamReader reader = new InputStreamReader(stream, "UTF8");
        StringWriter writer = new StringWriter();
        while (-1 != (n = reader.read(buffer))) writer.write(buffer, 0, n);
        return writer.toString();
    }

}

You get the data from the intent with getData() to get the URI to the file. Then you use the content resolver to open an input stream. You use the content resolver because you might not have direct access to the file otherwise. Thanks to this answer for the method to convert the input stream to a string.

You can add the following to the AndroidManifest.xml file inside the activity that has to open the file (pdf in our case)

        <intent-filter >
            <action android:name="android.intent.action.VIEW" />
            <category android:name="android.intent.category.DEFAULT" />
            <data android:mimeType="application/pdf" />
        </intent-filter>

Make sure you specify the proper mime format. The user will be prompted to choose your app if she wants to open the "pdf" file.

Check also Android intent filter: associate app with file extension for more details.

I think that you don't choose that. This is handle by the system. Same thing when you send an intent to be handled by a map, If you have skyfire for instance, the system pops up a window an you can choose the application and choose if you want it to always open this kind of file.
Unless of course if your application is the only one to open this kind of file.

Edit
I think if you want your app to say "hey I'm able to open these .cool files", you need to set up <intent-filters> with a tag <data> and specificy the mimeType or Uri. Check here for more details.