Fast way to export large amount of data in "Table" format
I started working on a package and LinkedLibrary achieving significant speedups with this job:
data = Table[{1. x, 1. y, 0.}, {x, 1000}, {y, 10^3}]~Flatten~1;
<< ExportTable`
AbsoluteTiming@ExportTable["test 1mio points.txt", data]
{0.0381513, Null}
You can find it here: https://github.com/Masterxilo/ExportTable
Here's an alternative: The package https://github.com/Masterxilo/PLYExport implements exporting of Points in PLY format, and extends Export with {"PLY", "BinaryFormat" -> True}
to use it, allowing to also export lots of points very quickly:
data = RandomReal[1., {10^6, 3}];
(*Graphics3D@Point[data]*)(*don't do this for large n: Use MeshLab or CloudCompare*)
AbsoluteTiming@
Export["test 1 mio points.ply",
Graphics3D@Point[data], {"PLY", "BinaryFormat" -> True}]
{0.308942, Null}
Not very fast, but acceptable.
data = RandomReal[1, {10^5, 3}];
Export["test1.txt", data, "Table"] // AbsoluteTiming
AbsoluteTiming[
str = Row[#, "\t"] & /@ data // Column // NumberForm[AccountingForm@#, 16] & // ToString;
Export["test2.txt", str, "String"]
]
Equal @@ (ReadList[#, {Number,Number,Number}] & /@ {"test1.txt","test2.txt"}) // AbsoluteTiming
I also noticed that the same code version 11.2 more slowly than 11.1