How to use \NewDocumentCommand

Short answers on your questions (feel free to ask for more details in the comments):

1. You should really define every function (inner or not) \protected (or robust) if it has to execute code that isn't expandable. So, use \cs_new_protected:Npn for non-expandable inner functions, even if they are only called inside a macro defined with \NewDocumentCommand (but see egreg's answer for a better description of when to define something protected or not, e.g., a macro like \multicolumn has to be expandable to place the \omit at the start of the cell, even though the next steps of processing are not expandable and do assignments).

2. Both document authors and package maintainers might use \NewDocumentCommand and friends. But it is good practice to only use it in the preamble, so not inside of \begin{document}...\end{document} (for the sake of structure and conventions, other than that there is nothing preventing you from using it later).

3. No, there aren't any problems with this as long as there are no naming clashes (which would throw an error anyways). If you're using LaTeX and packages like siunitx you're already mixing macros defined with both interfaces. Just keep in mind that \newcommand doesn't define robust commands, so the results could fail inside of an \edef context, while \NewDocumentCommand wouldn't (however there are interfaces in LaTeX to define robust commands without the use of xparse).

Whether to use \cs_new:Npn or \cs_new_protected:Npn (or variants thereof) depends on what the functions we're defining are supposed to do.

Any function that performs assignments of values to variables or of meaning to functions should be protected. For instance all new, set, clear or zero functions are protected and any function that uses them should also be.

However, it's possible to use protected functions also in “unprotected” ones, provided the external function doesn't itself perform assignments.

For instance, if you have a function \__cyker_purify:n that makes changes to its argument based on \tl_replace_all:Nn or \regex_replace_all:nnN, this has to be defined with \cs_new_protected:Npn. However, you're allowed to do

\cs_new:Npn \__cyker_process:n
 {
  \clist_map_function:nN { #1 } \__cyker_purify:n
 }

so \__cyker_process:n can be used inside another function that uses e or x expansion, say with

\tl_put_right:Nx \l__cyker_tablebody_tl { \__cyker_process:n { #1 } }

Finally, \NewDocumentCommand and \newcommand are simply different ways to define commands. There is no problem in mixing the two.