unexpected out of memory error on FreeBSD
I wrote a daemon that periodically scrapes information from a webpage that crashes after about 20 hours with the error: "out of memory (requested 1048576 bytes)". The program runs fine on my Ubuntu machine, but always crashes on FreeBSD even though there is plenty of memory to spare/memory stays constant/etc.
I boiled it down to the following code, which consistently crashes after about 4000-6000 iterations when compiled with -O2, and about 35000 without.
import Data.Conduit import Data.Conduit.List import qualified Data.ByteString as B import Network.HTTP.Conduit main :: IO () main = do manager <- newManager def loop manager 1 loop :: Manager -> Int -> IO () loop manager i = do putStrLn $ show i request <- parseUrl "http://localhost:8000/courselist.html" html <- runResourceT $ do response <- http request manager fmap B.concat $ responseBody response $$+- consume loop manager $ i + 1
For debugging, I served a copy of of a typical html sample from localhost using "python -m SimpleHTTPServer". The html file is attached.
... 35361 35362 35363 35364 35365 main: out of memory (requested 1048576 bytes)
$ uname -a FreeBSD oriskova 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243826: Tue Dec 4 06:55:39 UTC 2012 firstname.lastname@example.org:/usr/obj/usr/src/sys/GENERIC i386
$ gcc -v Using built-in specs. Target: i386-undermydesk-freebsd Configured with: FreeBSD/i386 system compiler Thread model: posix[[BR]] gcc version 4.2.1 20070831 patched [FreeBSD]
I wasn't sure if I should submit this or not since it seemed fairly specific to http-conduit, but the wiki said to go ahead and submit runtime errors, so that's what I did. This is my first time ever submitting a bug report and I tried my best to follow all the instructions at http://hackage.haskell.org/trac/ghc/wiki/ReportABug, but please tell me if there is something I'm doing wrong.