Are networks now faster than disks?
Solution 1:
Here are some numbers that you are probably looking for, as quoted by Jeff Dean, a Google Fellow:
Numbers Everyone Should Know
L1 cache reference 0.5 ns Branch mispredict 5 ns L2 cache reference 7 ns Mutex lock/unlock 100 ns (25) Main memory reference 100 ns Compress 1K bytes with Zippy 10,000 ns (3,000) Send 2K bytes over 1 Gbps network 20,000 ns Read 1 MB sequentially from memory 250,000 ns Round trip within same datacenter 500,000 ns Disk seek 10,000,000 ns Read 1 MB sequentially from network 10,000,000 ns Read 1 MB sequentially from disk 30,000,000 ns (20,000,000) Send packet CA->Netherlands->CA 150,000,000 ns
It's from his presentation titled Designs, Lessons and Advice from Building Large Distributed Systems and you can get it here:
- Dr Jeff Dean Keynote PDF or on slideshare.net
The talk was given at Large-Scale Distributed Systems and Middleware (LADIS) 2009.
Other Info
- Google Pro Tip: Use Back-Of-The-Envelope-Calculations To Choose The Best Design
- Stanford 295 Talk Software Engineering Advice from Building Large-Scale Distributed Systems
It's said that gcc -O4 emails your code to Jeff Dean for a rewrite.
Solution 2:
There are a lot of variables when it comes to network vs. disk, but in general, disk is faster.
The SATA 3.0 and SAS buses are 6 Gbps, vs. a networks 1Gbps minus protocol overhead. With RAID-10 15k SAS, the network is going to seem dog slow. In addition, you have disk cache and also the possibility of solid state harddrives, which depending on the scenario, could also increase speed. Random vs. Sequential data access plays a factor, as well as the block size in which data is being transferred. That all depends on the application that is being used to access the disk.
Now, I have not even touched on the fact that whatever you are transporting over the network is going to or coming from disk anyway...so.......again, disk is faster.
Solution 3:
Well, that depends on whether the network resource has the data you are requesting readily available (in memory or similar) or if it would just, in turn, read it from a disk.
In any case, throughput may be higher in some cases but I believe latency will be higher.
Solution 4:
The experience I have with this is that when you're on a 1Gbit connection and you try to download a file your harddisk is usually the bottleneck. A thing you have to keep in mind though is that you have to set up a connection first, which also takes time. So for sending big chunks of data network might actually be faster than disk.
Solution 5:
IMX the disk is still faster. The theoretical transfer rate of the network is high but in practice you don't get close to that.
About two years ago I had hard drive trouble on my laptop and the DMA went out. This made the hard drive dramatically slower, and in particular slower than network. But when I switched to another computer I was back to my original state of HDD faster than Internet.