Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • GHC GHC
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 5,249
    • Issues 5,249
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
  • Merge requests 581
    • Merge requests 581
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Releases
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Glasgow Haskell CompilerGlasgow Haskell Compiler
  • GHCGHC
  • Issues
  • #12758
Closed
Open
Issue created Oct 23, 2016 by Ben Gamari@bgamari🐢Maintainer

Bring sanity to our performance testsuite

The GHC testsuite's performance tests tends to be a constant source of busy-work as it requires that contributors manually bump performance numbers and propagate these changes across platforms. Moreover, the testsuite is poor at noticing performance regressions due to false positive failures (due to spurious environmental differences) and false negatives (due to the rather generous acceptance windows that many tests have).

Joachim and I briefly discussed this at Hac Phi and came up with the following proposal:

  • We rip expected performance numbers out of the .T files
  • We replace the existing compiler_stats_num_field test modifier with an implementation that dumps performance metrics to a CSV file
  • Introduce a tool to compare this CSV file to metrics associated with a ancestor commit via git notes
  • The tool would also be able to add notes
Edited Mar 10, 2019 by Ben Gamari
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking