Skip to content

unexpected out of memory error on FreeBSD

I wrote a daemon that periodically scrapes information from a webpage that crashes after about 20 hours with the error: "out of memory (requested 1048576 bytes)". The program runs fine on my Ubuntu machine, but always crashes on FreeBSD even though there is plenty of memory to spare/memory stays constant/etc.

I boiled it down to the following code, which consistently crashes after about 4000-6000 iterations when compiled with -O2, and about 35000 without.

import Data.Conduit
import Data.Conduit.List
import qualified Data.ByteString as B
import Network.HTTP.Conduit

main :: IO ()
main = do
    manager <- newManager def
    loop manager 1

loop :: Manager -> Int -> IO ()
loop manager i = do
    putStrLn $ show i
    request <- parseUrl "http://localhost:8000/courselist.html"
    html <- runResourceT $ do
        response <- http request manager
        fmap B.concat $ responseBody response $$+- consume
    loop manager $ i + 1

For debugging, I served a copy of of a typical html sample from localhost using "python -m SimpleHTTPServer". The html file is attached.

Sample Output:[[BR]]

...
35361
35362
35363
35364
35365
main: out of memory (requested 1048576 bytes)

Other information:

$ uname -a
FreeBSD oriskova 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243826: Tue Dec  4 06:55:39 UTC 2012     root@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
$ gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix[[BR]]
gcc version 4.2.1 20070831 patched [FreeBSD]

I wasn't sure if I should submit this or not since it seemed fairly specific to http-conduit, but the wiki said to go ahead and submit runtime errors, so that's what I did. This is my first time ever submitting a bug report and I tried my best to follow all the instructions at http://hackage.haskell.org/trac/ghc/wiki/ReportABug, but please tell me if there is something I'm doing wrong.

Trac metadata
Trac field Value
Version 7.4.2
Type Bug
TypeOfFailure OtherFailure
Priority normal
Resolution Unresolved
Component Runtime System
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information