1. 17 Mar, 2020 1 commit
  2. 19 Dec, 2019 1 commit
    • Moritz Kiefer's avatar
      Avoid race condition in hDuplicateTo · fad866e0
      Moritz Kiefer authored
      In our codebase we have some code along the lines of
      
      ```
      newStdout <- hDuplicate stdout
      stderr `hDuplicateTo` stdout
      ```
      
      to avoid stray `putStrLn`s from corrupting a protocol (LSP) that is
      run over stdout.
      
      On CI we have seen a bunch of issues where `dup2` returned `EBUSY` so
      this fails with `ResourceExhausted` in Haskell.
      
      I’ve spent some time looking at the docs for `dup2` and the code in
      `base` and afaict the following race condition is being triggered
      here:
      
      1. The user calls `hDuplicateTo stderr stdout`.
      2. `hDuplicateTo` calls `hClose_help stdout_`, this closes the file
      handle for stdout.
      3. The file handle for stdout is now free, so another thread
      allocating a file might get stdout.
      4. If `dup2` is called while `stdout` (now pointing to something
      else) is half-open, it returns EBUSY.
      
      I think there might actually be an even worse case where `dup2` is run
      after FD 1 is fully open again. In that case, you will end up not just
      redirecting the original stdout to stderr but also the whatever
      resulted in that file handle being allocated.
      
      As far as I can tell, `dup2` takes care of closing the file handle
      itself so there is no reason to do this in `hDuplicateTo`. So this PR
      replaces the call to `hClose_help` by the only part of `hClose_help`
      that we actually care about, namely, `flushWriteBuffer`.
      
      I tested this on our codebase fairly extensively and haven’t been able
      to reproduce the issue with this patch.
      fad866e0
  3. 14 Feb, 2019 1 commit
  4. 21 Aug, 2018 1 commit
    • Alec Theriault's avatar
      Fix ambiguous/out-of-scope Haddock identifiers · ddffa0cd
      Alec Theriault authored
      This drastically cuts down on the number of Haddock warnings when making
      docs for `base`.  Plus this means more actual links end up in the docs!
      Also fixed other small mostly markup issues in the documentation along
      the way.
      
      This is a docs-only change.
      
      Reviewers: hvr, bgamari, thomie
      
      Reviewed By: thomie
      
      Subscribers: thomie, rwbarton, carter
      
      Differential Revision: https://phabricator.haskell.org/D5055
      ddffa0cd
  5. 29 Apr, 2017 1 commit
  6. 26 Feb, 2017 1 commit
  7. 06 Jun, 2016 1 commit
  8. 23 Dec, 2015 1 commit
    • Eric Seidel's avatar
      Allow CallStacks to be frozen · 380b25ea
      Eric Seidel authored
      This introduces "freezing," an operation which prevents further
      locations from being appended to a CallStack.  Library authors may want
      to prevent CallStacks from exposing implementation details, as a matter
      of hygiene. For example, in
      
      ```
      head [] = error "head: empty list"
      
      ghci> head []
      *** Exception: head: empty list
      CallStack (from implicit params):
        error, called at ...
      ```
      
      including the call-site of `error` in `head` is not strictly necessary
      as the error message already specifies clearly where the error came
      from.
      
      So we add a function `freezeCallStack` that wraps an existing CallStack,
      preventing further call-sites from being pushed onto it. In other words,
      
      ```
      pushCallStack callSite (freezeCallStack callStack) = freezeCallStack callStack
      ```
      
      Now we can define `head` to not produce a CallStack at all
      
      ```
      head [] =
        let ?callStack = freezeCallStack emptyCallStack
        in error "head: empty list"
      
      ghci> head []
      *** Exception: head: empty list
      CallStack (from implicit params):
        error, called at ...
      ```
      
      ---
      
      1. We add the `freezeCallStack` and `emptyCallStack` and update the
         definition of `CallStack` to support this functionality.
      
      2. We add `errorWithoutStackTrace`, a variant of `error` that does not
         produce a stack trace, using this feature. I think this is a sensible
         wrapper function to provide in case users want it.
      
      3. We replace uses of `error` in base with `errorWithoutStackTrace`. The
         rationale is that base does not export any functions that use CallStacks
         (except for `error` and `undefined`) so there's no way for the stack
         traces (from Implicit CallStacks) to include user-defined functions.
         They'll only contain the call to `error` itself. As base already has a
         good habit of providing useful error messages that name the triggering
         function, the stack trace really just adds noise to the error. (I don't
         have a strong opinion on whether we should include this third commit,
         but the change was very mechanical so I thought I'd include it anyway in
         case there's interest)
      
      4. Updates tests in `array` and `stm` submodules
      
      Test Plan: ./validate, new test is T11049
      
      Reviewers: simonpj, nomeata, goldfire, austin, hvr, bgamari
      
      Reviewed By: simonpj
      
      Subscribers: thomie
      
      Projects: #ghc
      
      Differential Revision: https://phabricator.haskell.org/D1628
      
      GHC Trac Issues: #11049
      380b25ea
  9. 17 Dec, 2015 1 commit
    • thomie's avatar
      IO Handles: update comments [skip ci] · 1b6323b3
      thomie authored
      * hSetEcho, hGetEcho and hIsTerminalDevice are part of the Haskell2010
      report (but not Haskell98)
      
      * there are great `Note`s in GHC.IO.Handle.Types. Link to them.
      1b6323b3
  10. 16 Dec, 2015 1 commit
    • Herbert Valerio Riedel's avatar
      Start using `-W` instead of `-f(no-)warn` in some places · 437ebdda
      Herbert Valerio Riedel authored
      This replaces some occurences of `-f(no-)warn` with the new `-W`-aliases
      introduced via 2206fa8c / #11218, in cases which are guaranteed
      to be invoked with recent enough GHC (i.e. the stage1+ GHC).
      
      After this commit, mostly the compiler and the testsuite remain using
      `-f(wo-)warn...` because the compiler needs to be bootstrappable with
      older GHCs, while for the testsuite it's convenient to be able to quickly
      compare the behavior to older GHCs (which may not support the new flags yet).
      
      The compiler-part can be updated to use the new flags once GHC 8.3 development
      starts.
      
      Reviewed By: quchen
      
      Differential Revision: https://phabricator.haskell.org/D1637
      437ebdda
  11. 15 Dec, 2015 1 commit
  12. 24 Sep, 2014 1 commit
  13. 21 Sep, 2014 1 commit
  14. 18 Sep, 2014 1 commit
  15. 18 Nov, 2011 1 commit
  16. 26 Oct, 2011 1 commit
  17. 18 Jun, 2011 1 commit
  18. 24 May, 2011 1 commit
  19. 28 Jan, 2011 1 commit
  20. 01 Dec, 2010 1 commit
  21. 25 Nov, 2010 2 commits
  22. 24 Nov, 2010 1 commit
  23. 14 Jul, 2010 1 commit
  24. 02 Jul, 2010 1 commit
  25. 29 Jun, 2010 1 commit
  26. 14 May, 2010 1 commit
  27. 12 Jan, 2010 1 commit
  28. 23 Sep, 2009 1 commit
  29. 15 Jul, 2009 2 commits
  30. 13 Jul, 2009 1 commit
  31. 10 Jul, 2009 1 commit
  32. 23 Jun, 2009 1 commit
  33. 14 Jun, 2009 1 commit
    • Simon Marlow's avatar
      Save and restore the codec state when re-decoding · 1e74d272
      Simon Marlow authored
      We previously had an ugly hack to check for a BOM when re-decoding
      some binary data in flushCharBuffer.  The hack was there essentially
      because codecs like UTF-16 have a state, and we had not restored it.
      This patch gives codecs an explicit state, and implemented
      saving/restoring of the state as necessary.  Hence, the hack in
      flushCharBuffer is replaced by a more general mechanism that works for
      any codec with state.
      
      Unfortunately, iconv doesn't give us a way to save and restore the
      state, so this is currently only implemented for the built-in codecs.
      1e74d272
  34. 16 Jun, 2009 1 commit
  35. 12 Jun, 2009 1 commit
    • Simon Marlow's avatar
      Rewrite of the IO library, including Unicode support · 7b067f2d
      Simon Marlow authored
      Highlights:
      
      * Unicode support for Handle I/O:
      
        ** Automatic encoding and decoding using a per-Handle encoding.
      
        ** The encoding defaults to the locale encoding (only on Unix 
           so far, perhaps Windows later).
      
        ** Built-in UTF-8, UTF-16 (BE/LE), and UTF-32 (BE/LE) codecs.
      
        ** iconv-based codec for other encodings on Unix
      
      * Modularity: the low-level IO interface is exposed as a type class
        (GHC.IO.IODevice) so you can build your own low-level IO providers and
        make Handles from them.
      
      * Newline translation: instead of being Windows-specific wired-in
        magic, the translation from \r\n -> \n and back again is available
        on all platforms and is configurable for reading/writing
        independently.
      
      
      Unicode-aware Handles
      ~~~~~~~~~~~~~~~~~~~~~
      
      This is a significant restructuring of the Handle implementation with
      the primary goal of supporting Unicode character encodings.
      
      The only change to the existing behaviour is that by default, text IO
      is done in the prevailing locale encoding of the system (except on
      Windows [1]).  
      
      Handles created by openBinaryFile use the Latin-1 encoding, as do
      Handles placed in binary mode using hSetBinaryMode.
      
      We provide a way to change the encoding for an existing Handle:
      
         GHC.IO.Handle.hSetEncoding :: Handle -> TextEncoding -> IO ()
      
      and various encodings (from GHC.IO.Encoding):
      
         latin1,
         utf8,
         utf16, utf16le, utf16be,
         utf32, utf32le, utf32be,
         localeEncoding,
      
      and a way to lookup other encodings:
      
         GHC.IO.Encoding.mkTextEncoding :: String -> IO TextEncoding
      
      (it's system-dependent whether the requested encoding will be
      available).
      
      We may want to export these from somewhere more permanent; that's a
      topic for a future library proposal.
      
      Thanks to suggestions from Duncan Coutts, it's possible to call
      hSetEncoding even on buffered read Handles, and the right thing
      happens.  So we can read from text streams that include multiple
      encodings, such as an HTTP response or email message, without having
      to turn buffering off (though there is a penalty for switching
      encodings on a buffered Handle, as the IO system has to do some
      re-decoding to figure out where it should start reading from again).
      
      If there is a decoding error, it is reported when an attempt is made
      to read the offending character from the Handle, as you would expect.
      
      Performance varies.  For "hGetContents >>= putStr" I found the new
      library was faster on my x86_64 machine, but slower on an x86.  On the
      whole I'd expect things to be a bit slower due to the extra
      decoding/encoding, but probabaly not noticeably.  If performance is
      critical for your app, then you should be using bytestring and text
      anyway.
      
      [1] Note: locale encoding is not currently implemented on Windows due
      to the built-in Win32 APIs for encoding/decoding not being sufficient
      for our purposes.  Ask me for details.  Offers of help gratefully
      accepted.
      
      
      Newline Translation
      ~~~~~~~~~~~~~~~~~~~
      
      In the old IO library, text-mode Handles on Windows had automatic
      translation from \r\n -> \n on input, and the opposite on output.  It
      was implemented using the underlying CRT functions, which meant that
      there were certain odd restrictions, such as read/write text handles
      needing to be unbuffered, and seeking not working at all on text
      Handles.
      
      In the rewrite, newline translation is now implemented in the upper
      layers, as it needs to be since we have to perform Unicode decoding
      before newline translation.  This means that it is now available on
      all platforms, which can be quite handy for writing portable code.
      
      For now, I have left the behaviour as it was, namely \r\n -> \n on
      Windows, and no translation on Unix.  However, another reasonable
      default (similar to what Python does) would be to do \r\n -> \n on
      input, and convert to the platform-native representation (either \r\n
      or \n) on output.  This is called universalNewlineMode (below).
      
      The API is as follows.  (available from GHC.IO.Handle for now, again
      this is something we will probably want to try to get into System.IO
      at some point):
      
      -- | The representation of a newline in the external file or stream.
      data Newline = LF    -- ^ "\n"
                   | CRLF  -- ^ "\r\n"
                   deriving Eq
      
      -- | Specifies the translation, if any, of newline characters between
      -- internal Strings and the external file or stream.  Haskell Strings
      -- are assumed to represent newlines with the '\n' character; the
      -- newline mode specifies how to translate '\n' on output, and what to
      -- translate into '\n' on input.
      data NewlineMode 
        = NewlineMode { inputNL :: Newline,
                          -- ^ the representation of newlines on input
                        outputNL :: Newline
                          -- ^ the representation of newlines on output
                       }
                   deriving Eq
      
      -- | The native newline representation for the current platform
      nativeNewline :: Newline
      
      -- | Map "\r\n" into "\n" on input, and "\n" to the native newline
      -- represetnation on output.  This mode can be used on any platform, and
      -- works with text files using any newline convention.  The downside is
      -- that @readFile a >>= writeFile b@ might yield a different file.
      universalNewlineMode :: NewlineMode
      universalNewlineMode  = NewlineMode { inputNL  = CRLF, 
                                            outputNL = nativeNewline }
      
      -- | Use the native newline representation on both input and output
      nativeNewlineMode    :: NewlineMode
      nativeNewlineMode     = NewlineMode { inputNL  = nativeNewline, 
                                            outputNL = nativeNewline }
      
      -- | Do no newline translation at all.
      noNewlineTranslation :: NewlineMode
      noNewlineTranslation  = NewlineMode { inputNL  = LF, outputNL = LF }
      
      
      -- | Change the newline translation mode on the Handle.
      hSetNewlineMode :: Handle -> NewlineMode -> IO ()
      
      
      
      IO Devices
      ~~~~~~~~~~
      
      The major change here is that the implementation of the Handle
      operations is separated from the underlying IO device, using type
      classes.  File descriptors are just one IO provider; I have also
      implemented memory-mapped files (good for random-access read/write)
      and a Handle that pipes output to a Chan (useful for testing code that
      writes to a Handle).  New kinds of Handle can be implemented outside
      the base package, for instance someone could write bytestringToHandle.
      A Handle is made using mkFileHandle:
      
      -- | makes a new 'Handle'
      mkFileHandle :: (IODevice dev, BufferedIO dev, Typeable dev)
                    => dev -- ^ the underlying IO device, which must support
                           -- 'IODevice', 'BufferedIO' and 'Typeable'
                    -> FilePath
                           -- ^ a string describing the 'Handle', e.g. the file
                           -- path for a file.  Used in error messages.
                    -> IOMode
                           -- ^ The mode in which the 'Handle' is to be used
                    -> Maybe TextEncoding
                           -- ^ text encoding to use, if any
                    -> NewlineMode
                           -- ^ newline translation mode
                    -> IO Handle
      
      This also means that someone can write a completely new IO
      implementation on Windows based on native Win32 HANDLEs, and
      distribute it as a separate package (I really hope somebody does
      this!).
      
      This restructuring isn't as radical as previous designs.  I haven't
      made any attempt to make a separate binary I/O layer, for example
      (although hGetBuf/hPutBuf do bypass the text encoding and newline
      translation).  The main goal here was to get Unicode support in, and
      to allow others to experiment with making new kinds of Handle.  We
      could split up the layers further later.
      
      
      API changes and Module structure
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      
      NB. GHC.IOBase and GHC.Handle are now DEPRECATED (they are still
      present, but are just re-exporting things from other modules now).
      For 6.12 we'll want to bump base to version 5 and add a base4-compat.
      For now I'm using #if __GLASGOW_HASKEL__ >= 611 to avoid deprecated
      warnings.
      
      I split modules into smaller parts in many places.  For example, we
      now have GHC.IORef, GHC.MVar and GHC.IOArray containing the
      implementations of IORef, MVar and IOArray respectively.  This was
      necessary for untangling dependencies, but it also makes things easier
      to follow.
      
      The new module structurue for the IO-relatied parts of the base
      package is:
      
      GHC.IO
         Implementation of the IO monad; unsafe*; throw/catch
      
      GHC.IO.IOMode
         The IOMode type
      
      GHC.IO.Buffer
         Buffers and operations on them
      
      GHC.IO.Device
         The IODevice and RawIO classes.
      
      GHC.IO.BufferedIO
         The BufferedIO class.
      
      GHC.IO.FD
         The FD type, with instances of IODevice, RawIO and BufferedIO.
      
      GHC.IO.Exception
         IO-related Exceptions
      
      GHC.IO.Encoding
         The TextEncoding type; built-in TextEncodings; mkTextEncoding
      
      GHC.IO.Encoding.Types
      GHC.IO.Encoding.Iconv
      GHC.IO.Encoding.Latin1
      GHC.IO.Encoding.UTF8
      GHC.IO.Encoding.UTF16
      GHC.IO.Encoding.UTF32
         Implementation internals for GHC.IO.Encoding
      
      GHC.IO.Handle
         The main API for GHC's Handle implementation, provides all the Handle
         operations + mkFileHandle + hSetEncoding.
      
      GHC.IO.Handle.Types
      GHC.IO.Handle.Internals
      GHC.IO.Handle.Text
         Implementation of Handles and operations.
      
      GHC.IO.Handle.FD
         Parts of the Handle API implemented by file-descriptors: openFile,
         stdin, stdout, stderr, fdToHandle etc.
      7b067f2d