Draft: Fix #8045 - Move I/O manager benchmarks into the GHC tree
This MR replaces !4955 (closed).
The following points are relevant for this MR:
-
Relevant changes in the base library have been made for Event-related modules
-
All original benchmark programs have been included in the branch for comparison as the some of these required extensive updates/changes for them to be able to work with the newer GHC versions and/or libraries.
-
The following libraries need to be locally installed:
- network-v3.1.2.1
- attoparsec-0.14.1
- ghc-prim-0.7.0
- hashable-1.3.2.0
- integer-logarithms-1.0.3.1
- primitive-0.7.1.0
- scientific-0.3.7.0
and minor changes need to be made in the above libraries for compatibility with GHC 9+. The steps outlined at https://gitlab.haskell.org/ghc/ghc/-/wikis/debugging/installing-packages-inplace under the plan B heading are useful. After this, the makefile can with
make
and the benchmarks.cabal file can be run as:cabal build --with-compiler=$topdir/inplace/bin/ghc-stage2 --package-db=$topdir/inplace/lib/package.conf.d
-
All Bash shell and Python scripts have been updated for Python 3.8.5 and Bash 5+
-
Example outputs for the bench-timers.sh and bench-thread-delay.sh scripts have been added for reference.
Merge request reports
Activity
mentioned in issue #8045
added 1 commit
- 8eb25aef - Change network submodule url to GitLab copy (will need to be updated later)
added 87 commits
-
8eb25aef...633bbc1f - 35 commits from branch
master
- 686e4dc1 - Modify Event module exports
- 4cc733d7 - Modify Event.Poll module exports
- 44d971a9 - Modify .cabal for base library to expose Event-related modules
- 46384efe - Move GHC.Event.PSQ module out of os conditional block to avoid compilation errors
- 495d9330 - Add LICENSE for benchmarking utilities
- 1551838d - Add Timers benchmarking utility
- ed99351f - Timers: Update imports; add Semigroup instance for Config
- 775c8de4 - Add Args benchmarking utility
- 0a1ccc0d - Args: Clean up imports, code clean up
- e8d7badf - Add C file for DeadConn benchmarking utility
- e0a27be8 - Add DeadConn benchmarking utility
- e3d12dc7 - DeadConn: Add Semigroup instance for Monoid Config; update Socket function calls
- 1c136363 - Add EventFile benchmarking utility
- bc1379b8 - EventFile: Update imports; make minor formatting improvements
- 6fd60cc1 - Add EventHttp benchmarking utility
- f1eeeb94 - EventHttp: Update imports, Socket functions, variables to avoid unuse, shadow errors
- fc4b5d75 - Add EventUtil benchmarking utility
- 6f7bfb85 - EventUtil: Improve exception-catching function type signatures
- 452c73f5 - Add EventSocket benchmarking utility
- c22356fe - EventSocket: Update imports, functions, types for Socket-related functions
- 4bcfe90f - Add HttpClient benchmarking utility
- 0d2d9922 - HttpClient: Update function names, imports, time types, Config Semigroup instance
- da2b1171 - Add IntMap benchmarking utility
- 19ee3c58 - IntMap: Update import names
- 355101df - IntMap: Update function for correct IntMap insertWith return type
- d1065c54 - Add PongServer benchmarking utility
- 3138b72e - PongServer: Update imports, Socket functions, Semigroup instance for Config
- bfbe6b55 - Add PSQ benchmarking utility
- e8cb0750 - PSQ: Update module imports for GHC.Event
- 54664ffe - Add NoPush interface benchmarking utility
- 43cb13bd - NoPush: Update imports, Socket type and withFdSocket function
- d173b440 - Add RFC benchmarking utility
- e828dc93 - RFC: Update imports, qualify function calls with module names
- bcfa1ef8 - Add Signal benchmarking utility
- a77c756c - Export emState record field to manage (read for Signal) EventManager state
- bcc32b96 - Signal: Update imports, functions to manage signals (System.Posix.Signals)
- a30bff83 - Add Simple benchmarking utility
- 44d07182 - Simple: Update imports, Semigroup instance for Config
- 843b8ad0 - Add StaticHttp benchmarking utility
- ecac08a2 - StaticHttp: Update imports, functions for Socket, string append
- 89f61238 - Add ThreadDelay benchmarking utility
- 4bade9bf - ThreadDelay: Update imports, add Semigroup instance for Config
- 75cea86e - Add updated Python script for setting up benchmarking environment via using sysctl
- 41959241 - Add original bench-thread-delay shell script
- 355a46f0 - Add original bench-timers shell script
- 755b312a - Add .cabal for building benchmark executables
- 6bc32008 - Add updated Makefile for generating benchmark executables
- 13491e0c - Update Makefile targets for cleaning up data, program executables
- 019c10f8 - Update gnuplot options to generate events vs. threads benchmarking plot
- 5b17fdcc - Update gnuplot options to generate thread-delay benchmarking plot for GHC I/O manager
- b35ca958 - An example output for the bench-timers.sh script
- d3cafa72 - An example output for the bench-thread-delay.sh script
Toggle commit list-
8eb25aef...633bbc1f - 35 commits from branch
added 24 commits
- 526c9505 - PongServer: Update imports, Socket functions, Semigroup instance for Config
- 1f95af2e - Add PSQ benchmarking utility
- e4d1d859 - PSQ: Update module imports for GHC.Event
- 645cff7d - Add NoPush interface benchmarking utility
- ab77d308 - NoPush: Update imports, Socket type and withFdSocket function
- 5dea97b6 - Add RFC benchmarking utility
- 7d6143d6 - RFC: Update imports, qualify function calls with module names
- f19486b3 - Add Signal benchmarking utility
- 5b7e996a - Export emState record field to manage (read for Signal) EventManager state
- 825077bb - Signal: Update imports, functions to manage signals (System.Posix.Signals)
- 601e8650 - Add Simple benchmarking utility
- e08ccf17 - Simple: Update imports, Semigroup instance for Config
- 87a778c9 - Add StaticHttp benchmarking utility
- 3f3aa978 - StaticHttp: Update imports, functions for Socket, string append
- 93fdfde2 - Add ThreadDelay benchmarking utility
- 2914c1da - ThreadDelay: Update imports, add Semigroup instance for Config
- 4f42d86a - Add updated Python script for setting up benchmarking environment via using sysctl
- 22625615 - Add original bench-thread-delay shell script
- 94108d62 - Add original bench-timers shell script
- ae78f6b5 - Add .cabal for building benchmark executables
- a92b3068 - Add updated Makefile for generating benchmark executables
- 01fb4f16 - Update Makefile targets for cleaning up data, program executables
- c6170c46 - Update gnuplot options to generate events vs. threads benchmarking plot
- 36b775a8 - Update gnuplot options to generate thread-delay benchmarking plot for GHC I/O manager
Toggle commit listadded 1 commit
- 332ee342 - testsuite: Use appropriate threadDelay in T16916
- Resolved by Alina Banerjee
417 417 else 418 418 exposed-modules: 419 419 GHC.Event 420 GHC.Event.Manager 421 GHC.Event.Poll 422 GHC.Event.Thread 423 GHC.Event.TimerManager Annoyingly, I'm not sure whether we can just export these modules without involving the @core-libraries committee...
I approve of exporting them. @chessai you agree?
I'm not so eager to expose more internal GHC state to users. It's also not quite clear to me why a benchmark that's supposed to be testing external interfaces needs internal interfaces that most users today can't access.
If these need to be exported then I argue that a GHC API entry need to be defined for them so these internal interfaces don't become ABI.
changed this line in version 8 of the diff
- Resolved by Alina Banerjee
- Resolved by Alina Banerjee
I'm afraid I can't review any of the specifics here. Maybe @Phyx can do a better job of reviewing here or appoint someone who can?
We recently moved a number a benchmarks that depend on external packages out of the tree. So I'm not sure if that's the right step?
@mpickering looked into that. Maybe he can comment.
I agree with you here. I don't like increasing the dependencies on GHC/Base. In particular I'm weary about the network dependency.
Most Haskell packages have a long, long dependency chain and these not only have an impact on compile time but but they also have an impact on debugging. This adds an aweful lot of dependencies..
Is there any reason these can't leave out of tree like nofib?
@Phyx As described in the original issue #8045:
When we developed the scalable I/O manager (i.e. the first version that used epoll) we wrote a bunch of benchmarks that were never moved to the GHC tree or integrated into the make system:
It would be nice to move this into the GHC tree to catch future regressions and use them when trying to make improvements.
the benchmarks have been put in the GHC tree. A decision on whether they should be left in tree or otherwise needs to be made. Since @mpickering had last commented on the original issue, his input would help?
Edited by Alina Banerjee
- Resolved by Alina Banerjee
added 1 commit
- f0276597 - Move event benchmarks to utils/benchmarks/events/
added 144 commits
-
f0276597...41d6cfc4 - 93 commits from branch
master
- 68cd40a1 - Modify Event module exports
- 0a695ab0 - Modify Event.Poll module exports
- 630da3d0 - Add LICENSE for benchmarking utilities
- bf816128 - Add Timers benchmarking utility
- 23be8460 - Timers: Update imports; add Semigroup instance for Config
- 91511464 - Change Event.TimerManager looping function name to avoid clash with Event.Manager function
- 527fc71b - Add Args benchmarking utility
- 02fd8eb4 - Args: Clean up imports, code clean up
- b3343a80 - Add C file for DeadConn benchmarking utility
- 075df5c7 - Add DeadConn benchmarking utility
- 9c36288b - DeadConn: Add Semigroup instance for Monoid Config; update Socket function calls
- 88900a38 - Add EventFile benchmarking utility
- 94ea2991 - EventFile: Update imports; make minor formatting improvements
- 6107ae67 - Add EventHttp benchmarking utility
- 665dbe9e - EventHttp: Update imports, Socket functions, variables to avoid unuse, shadow errors
- 2c22e6d8 - Add EventUtil benchmarking utility
- 99763fac - EventUtil: Improve exception-catching function type signatures
- 20b081de - Add EventSocket benchmarking utility
- 03ac18af - EventSocket: Update imports, functions, types for Socket-related functions
- 105c0f62 - Add HttpClient benchmarking utility
- c9e887a9 - HttpClient: Update function names, imports, time types, Config Semigroup instance
- 3d9b165c - Add IntMap benchmarking utility
- df024842 - IntMap: Update import names
- 2fd62ea9 - IntMap: Update function for correct IntMap insertWith return type
- ec591dac - Add PongServer benchmarking utility
- dc118844 - PongServer: Update imports, Socket functions, Semigroup instance for Config
- dff19b19 - Add PSQ benchmarking utility
- 4d4f91ea - PSQ: Update module imports for GHC.Event
- ed53a5bf - Add NoPush interface benchmarking utility
- a78f8eb1 - NoPush: Update imports, Socket type and withFdSocket function
- 6a0571df - Add RFC benchmarking utility
- fff80181 - RFC: Update imports, qualify function calls with module names
- 94aa0820 - Add Signal benchmarking utility
- efcc06b9 - Export emState record field to manage (read for Signal) EventManager state
- e01e16a7 - Signal: Update imports, functions to manage signals (System.Posix.Signals)
- 825bd5b9 - Add Simple benchmarking utility
- a0b2706c - Simple: Update imports, Semigroup instance for Config
- 91f486c5 - Add StaticHttp benchmarking utility
- 55d5b836 - StaticHttp: Update imports, functions for Socket, string append
- 9fcd63a3 - Add ThreadDelay benchmarking utility
- 20e11f3e - ThreadDelay: Update imports, add Semigroup instance for Config
- 2c39d37a - Add updated Python script for setting up benchmarking environment via using sysctl
- c00f096d - Add original bench-thread-delay shell script
- 9ec5caaf - Add original bench-timers shell script
- 445bc4a4 - Add .cabal for building benchmark executables
- 7ca72263 - Add updated Makefile for generating benchmark executables
- c841af50 - Update Makefile targets for cleaning up data, program executables
- aa5ba1f8 - Update gnuplot options to generate events vs. threads benchmarking plot
- b9d9b40a - Update gnuplot options to generate thread-delay benchmarking plot for GHC I/O manager
- de9fedc3 - testsuite: Use appropriate threadDelay in T16916
- c708b969 - Move event benchmarks to utils/benchmarks/events/
Toggle commit list-
f0276597...41d6cfc4 - 93 commits from branch
@mpickering Your input is needed to finalize work on this issue related to the following:
- figuring out if the event benchmarks should be out of the GHC tree similar to nofib
- checking if the most recent change in exporting all necessary functions etc via
libraries/base/GHC/Event.hs
is the way to go - or if the GHC API needs to be used for access to entities in the Event module.
assigned to @alinab
Unless someone is actively working on #18380, it would be good to go over
GHC.Event.TimerManager
and find out refactor code as needed.