How te get bold + small caps with latin modern or computer modern

The log file shows you're using XeLaTeX on a “vanilla” TeX Live, but that the CMU Serif font is not available as a system font.

Running fc-cache -fsv by itself isn't sufficient, until you also make the TeX Live fonts available to the fontconfig library.

The two commands

> sudo cp $(kpsewhich -var-value TEXMFSYSVAR)/fonts/conf/texlive-fontconfig.conf /etc/fonts/conf.d/09-texlive.conf
> sudo fc-cache -fsv

should do what you need. With > I represent the shell prompt. The file we're copying as 09-texlive.conf in /etc/fonts/conf.d contains

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
  <dir>/usr/local/texlive/2016/texmf-dist/fonts/opentype</dir>
  <dir>/usr/local/texlive/2016/texmf-dist/fonts/truetype</dir>
  <dir>/usr/local/texlive/2016/texmf-dist/fonts/type1</dir>
</fontconfig>

but this is not really important. What's important that after doing that setup, all fonts in the TeX Live main tree will be available to fontconfig and so also XeTeX will be able to find them by font name rather than only by file name.

If that doesn't work, you can always call CMU Serif by file name:

\documentclass[11pt]{article}

\usepackage[tuenc]{fontspec}

\setmainfont{cmun}[
  Extension=.otf,
  UprightFont=*rm,
  ItalicFont=*ti,
  BoldFont=*bx,
  BoldItalicFont=*bi,
]

\begin{document}

not bold or caps \textsc{small caps}

\textbf{bold} \textbf{\textsc{bold small caps}}

\textit{italic} \textsc{\textit{italic small caps}}

\textbf{\textit{bold italic}} \textbf{\textit{\textsc{bold italic small caps}}}

\end{document}

Note that CMU Serif Italic and CMU Serif BoldItalic have no small caps glyphs, so you get normal italic and bold small caps in the third and fourth lines.

If the suggested procedure for fc-cache is successful, the code

\documentclass[11pt]{article}

\usepackage[tuenc]{fontspec}

\setmainfont{CMU Serif}

\begin{document}

not bold or caps \textsc{small caps}

\textbf{bold} \textbf{\textsc{bold small caps}}

\textit{italic} \textsc{\textit{italic small caps}}

\textbf{\textit{bold italic}} \textbf{\textit{\textsc{bold italic small caps}}}

\end{document}

should work with both XeLaTeX and LuaLaTeX.

Solution given here (for my ubuntu 2016)

Copy the texlive-fontconfig.conf to /etc/fonts/conf.d/09-texlive.conf

> sudo cp $(kpsewhich -var-value TEXMFSYSVAR)/fonts/conf/texlive-fontconfig.conf /etc/fonts/conf.d/09-texlive.conf

Modify /etc/fonts/conf.d/09-texlive.conf : (Source : http://www.tug.org/texlive/doc/texlive-en/texlive-en.html#x1-340003.4.4)

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
  <dir>/usr/local/texlive/2016/texmf-dist/fonts/opentype</dir>
  <dir>/usr/local/texlive/2016/texmf-dist/fonts/truetype</dir>
  <dir>/usr/local/texlive/2016/texmf-dist/fonts/type1</dir>
  <selectfont>
    <rejectfont>
      <glob>/usr/local/texlive/2016/texmf-dist/fonts/type1/public/cm-unicode/*</glob>
    </rejectfont>
  </selectfont>
</fontconfig>

Run

> sudo fc-cache -fsv

Test with :

\documentclass[11pt]{article}
\usepackage[tuenc]{fontspec}
\setmainfont{CMU Serif}
\begin{document}
not bold or caps \textsc{small caps} \par
\textbf{bold} \textbf{\textsc{bold small caps}} \par
\textit{italic} \textsc{\textit{italic small caps}} \par
\textbf{\textit{bold italic}} \textbf{\textit{\textsc{bold italic small caps}}}
\end{document}