rmarkdown pdf font not available

This option must be indented. It worked when you didn't indent the option latex_engine only because it was ignored and not really passed to pdf_document() as an argument. R Markdown uses the yaml package to parse YAML, and you can compare the output when latex_engine is indented or not:

yaml::yaml.load(
'output:
  pdf_document:
    latex_engine: xelatex
mainfont: roboto')

Output:

$output
$output$pdf_document
$output$pdf_document$latex_engine
[1] "xelatex"

$mainfont
[1] "roboto"

When not indented, you were essentially calling rmarkdown::render(, pdf_document()):

yaml::yaml.load(
'output:
  pdf_document:
  latex_engine: xelatex
mainfont: roboto')

Output:

$output
$output$pdf_document
NULL

$output$latex_engine
[1] "xelatex"


$mainfont
[1] "roboto"

In this case, pdf_document's default engine pdflatex is used.

I cannot say for sure, but I believe the roboto package is for pdflatex only. Your original example was failing because you mixed up pdflatex and xelatex: roboto is for pdflatex, and the mainfont option is for xelatex (which will be translated to LaTeX code \setmainfont{roboto} using the fontspec package). You can choose either way, but not both.

If you want to use xelatex, you have to make sure you have installed the font in your system, and you know the exact font name (case-sensitive). I guess the name is probably Roboto. Unless you have other reasons to use xelatex (e.g. you know the fontspec package well enough and want to configure more fonts), I'd recommend you to stay with the roboto package and pdflatex since it is simple enough and you don't have to learn too many lower-level technical details.

You may spend a minute reading this page about YAML: https://bookdown.org/yihui/bookdown/r-markdown.html

Update: Yihui's answer comprehensively addresses the question

This is a problem caused by the way RMarkdown is parsing the YAML header. The latex_engine should not be indented and then it will work. Currently, I believe pandoc is ignoring the latex_engine command because it is nested in pdf_output when it is part of the output part. Below is the correct header:

---
title: "My Title"
header-includes:
    - \usepackage[sfdefault]{roboto}
    - \usepackage[T1]{fontenc}
output:
    pdf_document:
    latex_engine: xelatex
mainfont: roboto
---