File read/write operations >= 2GiB fail on macOS with an "Invalid argument" exception
Summary
File reads/writes bigger than 2GB result in an "Invalid argument" exception on macOS. Files bigger than 2GB still work, but individual read/write operations bigger than 2GB fail.
This was already reported for the bytestring
library, but the actual problem is in base
. I didn't see an earlier report.
bytestring
report: https://github.com/haskell/bytestring/issues/153
Python has a workaround for this issue and more discussion: https://bugs.python.org/issue24658
Steps to reproduce
{-# LANGUAGE ScopedTypeVariables #-}
import Foreign.Ptr (Ptr)
import Foreign.Marshal.Alloc (mallocBytes, free)
import System.IO (hPutBuf, withBinaryFile, IOMode (WriteMode))
-- more than 2GiB
numBytes :: Int
numBytes = 2264375889
main :: IO ()
main = do
(ptr :: Ptr ()) <- mallocBytes numBytes
-- the next line produces the exception on macOS
withBinaryFile "test.out" WriteMode (\h -> hPutBuf h ptr numBytes)
free ptr
Expected behavior
The base
library provides operating system independent buffering and encoding functionality. I think it should work around operating system peculiarities and write all the data to the test.out
file without throwing an excepion.
Environment
Tested with GHC 8.6.5 on macOS 10.14.6
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information