Why is “/dev/rdisk” about 20 times faster than “/dev/disk” in Mac OS X

The accepted answer is right, but it doesn’t go into much detail.

One of the key differences between /dev/disk and /dev/rdisk, when you access them from user space, is that /dev/disk is buffered. The read/write path for /dev/disk breaks up the I/O into 4KB chunks, which it reads into the buffer cache, and then copies into the user space buffer (and then issues the next 4KB read…). This is nice in that you can do unaligned reads and writes, and it just works. In contrast, /dev/rdisk basically just passes the read or write straight to the device, which means the start and end of the I/O need to be aligned on sector boundaries.

If you do a read or write larger than one sector to /dev/rdisk, that request will be passed straight through. The lower layers may break it up (eg., USB breaks it up into 128KB pieces due to the maximum payload size in the USB protocol), but you generally can get bigger and more efficient I/Os. When streaming, like via dd, 128KB to 1MB are pretty good sizes to get near-optimal performance on current non-RAID hardware.

The caching being done by /dev/disk’s read and write paths is very simple and almost brain dead. It caches even if not strictly necessary; like if the device could memory map and directly transfer into your app's buffer. It does small (4KB) I/Os, which leads to a lot of per-I/O overhead. It does not do any read ahead or write behind.

From man hdiutil:

/dev/rdisk nodes are character-special devices, but are "raw" in the BSD sense and force block-aligned I/O. They are closer to the physical disk than the buffer cache. /dev/disk nodes, on the other hand, are buffered block-special devices and are used primarily by the kernel's filesystem code.

In layman's terms /dev/rdisk goes almost directly to disk and /dev/disk goes via a longer more expensive route

It seems /dev/disk and /dev/rdisk works different for HDDs and SSDs. Wanna checked it for MicroSD card. Just wrote 2GB disk image to Sandisk Ultra MicroSD 64GB (https://www.amazon.com/gp/product/B073JYVKNX).

Repeated tests several times, but results were stable: 17MB/s for /dev/disk vs 20MB/s for /dev/rdisk. Changing bs=1m to bs=16m gives absolutely no difference in writing speed.

1. Writing to /dev/disk2

   sudo dd if=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531.img of=/dev/disk2 bs=1m
   2094006272 bytes transferred in 121.860007 secs (17183704 bytes/sec)
   

2. Writing to /dev/rdisk2

   $ sudo dd if=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531.img of=/dev/rdisk2 bs=1m
   2094006272 bytes transferred in 102.743870 secs (20380839 bytes/sec)
   

Then I decided to test reading speed: 26MB/s for /dev/disk vs 87MB/s for /dev/rdisk. Changing bs=1m to bs=16m gives absolutely no difference in reading speed.

1. Reading from /dev/disk2

   sudo dd if=/dev/disk2 of=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531-2.img bs=1m
   257949696 bytes transferred in 9.895572 secs (26067184 bytes/sec)
   

2. Reading from /dev/rdisk2

   $ sudo dd if=/dev/rdisk2 of=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531.img bs=1m
   877658112 bytes transferred in 10.021974 secs (87573377 bytes/sec)