How can I create a github permalink to a line in a markdown .md or .Rmd file?

You can't link to a line number, because the line number anchors are provided by the GitHub code preview interface. Since, as you mention, markdown files are rendered differently, they don't have the line number anchors that you are looking for.

You can, however, link to a header (anything starting with one or more #s.) Hover over the header to make the link icon appear:

Then right click the link icon and "Copy Link Address".

If you are working inside the same markdown file as your target link, you can use the markdown anchor syntax: [link name](#anchor-name-with-hyphens).

Yes you can, link to specific lines in a Markdown file, by going to the Blame view of the file.

Then, press y to get a permalink to the version of the file that you see now. This will ensure the link persists if the file changes, is moved, renamed or deleted.

At this point, you can click a link, or Shift+click a range of lines, and the URL will update automatically.

Here is an example link.