ERROR: Uploading artifacts as "archive" to coordinator... 413 Request Entity Too Large

On our eventlog jobs the .tar.gz is too big to upload to the artifact server:

• Run with some eventlog filtering to make smaller eventlogs
• Bump the limit of the allowed artifact size on the server

This is quite high priority as it is breaking our performance monitoring infra - https://gitlab.haskell.org/ghc/head.hackage/-/jobs/1380727

Any other ideas? cc @RyanGlScott @teo @chreekat