In TeXstudio, how to compile only the subfile?

You can use a magic comment called %!TeX root at the beginning of your subfile.tex to override the default root detection by TXS. This way, when you are in subfile.tex and press compile, TXS will treat subfile.tex as root and compile that instead of main.tex, as required.

As an example, it might look something like:

subfile.tex

%!TeX root = subfile
\documentclass[main.tex]{subfiles}
\begin{document}
    Hello world.
\end{document}

This was also covered in the TXS user manual.

There is some kind of workaround one can use. You can write a small macro to run a single file using the subfiles class in TeXstudio. Actually it the macro doesn't care about the documentclass.

Suppose you have the following two files main-document.tex

\documentclass{article}
\usepackage{subfiles}
\begin{document}
  My main file text.
  \subfile{subfile-document}
\end{document}

and subfile-document.tex

\documentclass[main-document.tex]{subfiles}
\begin{document}
    Some subfile text
\end{document}

Now you can define a macro in TeXstudio using Script.

%SCRIPT
var filename = app.getCurrentFileName();
var dir = filename.substring(0, filename.lastIndexOf("/"));
var proc = system("pdflatex " + filename, dir);

The macro does the following:

• Get the current select filename
• Extract the directory path
• Call pdflatex <filename> inside the cmd and passes the correkt directory to the command line

TeXstudio uses keybindings for macros by default Shift + F1 but you are able to remap them. The number of the Fx key depend on the number of macros you have defined.

Now you can run the only the subfile by

1. Select the subfile-document.tex tab
2. Press Shift + F1 to run the macro. (TeXstudio might ask you for futherer privileges)

EDIT: I've created an issue asking about how to load a pdf file using a macro script on https://github.com/texstudio-org/

EDIT 2 You can use this updated script to display the built pdf using the inbuilt pdf viewer.

%SCRIPT

function quote(filename){
    return '"' + filename + '"';
}

var filename = app.getCurrentFileName();
var dir = filename.substring(0, filename.lastIndexOf("/"));
var pdflatexcmd = '../MikTex/texmfs/install/miktex/bin/x64/pdflatex.exe  -src -interaction=nonstopmode '
var proc = system(pdflatexcmd + quote(filename), dir);
proc.waitForFinished();

var pdfFilename = quote(filename.replace(/\.[^/.]+$/, ".pdf"));

app.runInternalCommand("txs:///view-pdf-internal" , "--embedded", pdfFilename);