The plainHttpTransport implementation saved the resulting file only when the
server responded with 200 OK, thus ignoring the payload from the server when
the server responded with a 206 Partial Content. This was causing incremental
updates through hackage-security to fail (it wouldn't fail fatally;
hackage-security would notice the invalid timestamp and download the whole
thing instead, but of course this means we're not actually getting any
incremental behaviour).

Function `putHttpFile` already allowed for request headers; we just the same
thing in `getHttp` as is done in `putHttpFile`.

The powershell script contains the password, so writing it to a file
is a bit unsafe. We know pass it via stdin.
This commit also improves the error message for network failures a
little bit.

Passing passwords via command line arguments is insecure: anyone who
is able to read the process list on the system can read the passwords
as well.

For curl, we pass the password via stdin by using the --config option,
which allows us to pass arbitrary additional options via stdin.

Unfortunately, wget's --config option does not support - for stdin. So
we instead use the --input-file option to pass an URI with the password
via stdin.

This fixes an issue in the curl transport when a text response is returned
that doesn't end with a newline from a POST or PUT request. The code expects
the HTTP status code to be on a new line, but that is not the case if the response
doesn't have a trailing new line. We fix this by always printing a new line before
printing the status code. To avoid duplicate new lines, we now strip trailing empty
lines at the end of the response.

This commit refactors the powershell transport to avoid duplication and
also fixes a few problems with it:

* We now run powershell with the ExecutionPolicy bypass, is required when
  the default security policy disallows executing unsigned scripts
  (like our script)
* The powershell script itself has been refactored to behave more like
  curl. In particular, it now prints HTTP status code errors in a nicer
  way instead of failing with a PowerShell Exception + backtrack. We also
  now only print the exception message when we get any other exception,
  since the user is likely not familar with the powershell script at all
  and the other information is thus only confusing and not helpful at all.
* We now handle the case where the server returns some message after a
  POST or a PUT correctly. Previously, that resulted in the raw bytes
  being written to stdout before the HTTP status code, which confused the
  simplistic parser. We now always write the HTTP status code first and
  decode the bytes as UTF8 before sending them to stdout as well.

This patch was tested with curl, wget and plain-http, but it's still untested on powershell.
The wget implementation will only work if the available wget supports the `--method` switch.

The initial patch would always try to use HTTPS, even when the repo
specified to use HTTP. This works for the central community hackage
but obviously does not work in general.

The new logic is that we always follow what is specified for the
remote repo in the config, except for built-in known repos (currently
just the central community hackage) where we mark them as also
supporting https. For upload when uploading to such a marked repo
then we will try https and will complain if the plain-http impl was
selected automatically (but it's ok if selected manually).

This patch also changes things so that for http urls on download, we
stick to the builtin http impl by default, and only use the external
ones if https support is required (i.e. because the repo was
configured to use an https url)

Move utils into other Util modules.
Reformat all code to 80 cols.
Reorder code and add more comments.
Use long form style program args, e.g. --silent rather than -s
Finish implementation of form upload with wget
Fix reporting of server error messages for upload (curl & builtin)
Implement collecting of ETags for curl and wget.
Fix wget for case of 304 not modified response (wget uses exit code 8).
Rework transport configuration phase.

U-CIQDEV\gbazerman authored May 01, 2015 and Duncan Coutts committed Jun 29, 2015

Supports both uploading and downloading.
Basic built-in HTTP is still supported.

This reverts commit c9a77353.

This reverts commit 03c134d5.

Committed by mistake.

nkaretnikov authored Jul 26, 2014 and Mikhail Glushenkov committed Jul 26, 2014

Fixes #1962.

Herbert Valerio Riedel authored May 01, 2014 and tibbe committed May 02, 2014

This makes the HTTP user-agent field look like so:

    User-Agent: cabal-install/1.21.0.0 (linux; x86_64)

This allows us to gather more statistics about our user-base as
with the current user-agent string we can only do a rough break-down by
cabal-install version numbers.

(cherry picked from commit bd0fcd55)

This makes the HTTP user-agent field look like so:

    User-Agent: cabal-install/1.21.0.0 (linux; x86_64)

This allows us to gather more statistics about our user-base as
with the current user-agent string we can only do a rough break-down by
cabal-install version numbers.

When the local and remote repositories match.

This means tweaking CPP conditionals and file pragmas to remove handling
of compilers other than GHC, and GHC versions before LANGUAGE pragmas
existed.

And use exceptions rather than return codes.