Subject: ByteStrings and the Terabyte barrier
Newsgroups: gmane.comp.lang.haskell.libraries
Date: 2006-05-15 00:17:36 GMT (3 years, 7 weeks, 3 days, 4 hours and 39 minutes ago)
Just for fun, I was able to filter a 1 Terabyte stream through a
Haskell program, using Data.ByteString.Lazy, with a constant 2M
heapsize, in
$ time ( repeat 100 ; cat /home/dons/data/10G | ./lazy > /dev/null )
12752.37s user 4069.35s system 58% cpu 8:03:11.42 total
8 hours.
Just so we can say it is possible :)
-- Don