Improving the Ruby GC
(this post has been moved over from lloydforge.org)
overview
This page details some changes to the ruby garbage collector which seem
to afford a 25% reduction in maximum heap memory usage, and nearly double
the amount of heap space ruby’s is able to reclaim. This comes at the
cost of a 2% performance hit. More to come, stay tuned.
patch
The latest patch to ruby 1.8.6 is available here.
summary of changes
- Fixed 2k heaps
- Heap implementation broken into separate c/h file
- heaps stored sorted by address for efficient is_pointer_to_heap()
- “adaptive GC frequency” - perform GC more frequently when free
percentage is high.
testing strategy
To test changes, I’m comparing the performance of a handful of tests
(most contributed by Hugh Sasse, many thanks!). Ruby 1.8.6 unpatched
is run over these tests, then compared against the performance of
ruby 1.8.6 with my patch. In order to get “maximum heap size”,
“current heap size”, and “nmber of gc passes”, a GC.heap_info
function has been hacked in to both.
NOTE: When referring to heap size, we’re really talking about the amount of
memory dedicated to ruby heaps, not the total heap usage of the ruby
process.
finally, all tests are performed on a macbook dual core with 2 gigs of
RAM. We use wallclock time, but I’ve ensured an idle machine and rerun
tests to verify that system load doesn’t significantly affect results.
data
| test | before | after | difference | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| test | time | gc passes | max mem | final mem | time | gc passes | max mem | final mem | time | gc passes % | max mem % | final mem % |
| cases/weak_hashes_read.rb | 0.459784 | 8 | 4473980 | 4473980 | 0.448108 | 9 | 2570240 | 2570240 | 97% | 112% | 57% | 57% |
| cases/create_hashes_yaml.rb | 25.913408 | 75 | 27300060 | 8253120 | 26.433873 | 75 | 18012160 | 614400 | 102% | 100% | 65% | 7% |
| cases/create_hashes.rb | 0.134596 | 4 | 1208000 | 560000 | 0.142204 | 6 | 1077248 | 75776 | 105% | 150% | 89% | 13% |
| cases/arrays_read_yaml.rb | 0.76034 | 14 | 4473980 | 4473980 | 0.752775 | 13 | 3278848 | 3278848 | 99% | 92% | 73% | 73% |
| cases/ostruct_read.rb | 2.64286 | 19 | 15055440 | 15055440 | 2.865675 | 22 | 8781824 | 8781824 | 108% | 115% | 58% | 58% |
| cases/create_ostructs_yaml.rb | 13.673743 | 55 | 27299780 | 15055440 | 14.471611 | 57 | 15616000 | 9125888 | 105% | 103% | 57% | 60% |
| cases/classes_read_yaml.rb | 0.825455 | 13 | 4473940 | 4473940 | 0.84994 | 13 | 4151296 | 4149248 | 102% | 100% | 92% | 92% |
| cases/create_weak_hashes2.rb | 0.497352 | 7 | 4473940 | 4473940 | 0.512148 | 9 | 2643968 | 2490368 | 102% | 128% | 59% | 55% |
| cases/create_arrays_yaml.rb | 9.002692 | 43 | 15055660 | 8253140 | 9.168935 | 44 | 9859072 | 2252800 | 101% | 102% | 65% | 27% |
| cases/create_arrays.rb | 0.170999 | 4 | 2374420 | 560000 | 0.17948 | 6 | 1912832 | 75776 | 104% | 150% | 80% | 13% |
| cases/growarray.rb | 1.933119 | 9 | 200000 | 200000 | 2.116156 | 9 | 102400 | 75776 | 109% | 100% | 51% | 37% |
| cases/arrays_read.rb | 0.089326 | 4 | 2374460 | 560000 | 0.092244 | 5 | 1679360 | 73728 | 103% | 125% | 70% | 13% |
| cases/hashes_read_yaml.rb | 1.982008 | 23 | 4473920 | 4473920 | 1.908458 | 20 | 4532224 | 4532224 | 96% | 86% | 101% | 101% |
| cases/plist.rb | 4.926278 | 23 | 15055760 | 15055760 | 4.773025 | 18 | 12734464 | 10463232 | 96% | 78% | 84% | 69% |
| cases/create_ostructs.rb | 3.076644 | 22 | 15055660 | 11276480 | 3.284315 | 24 | 8800256 | 92160 | 106% | 109% | 58% | 0% |
| cases/create_weak_hashes.rb | 1.205506 | 17 | 4473960 | 4473960 | 1.131734 | 14 | 5253120 | 4933632 | 93% | 82% | 117% | 110% |
| cases/classes_read.rb | 0.056913 | 3 | 1208000 | 1208000 | 0.062934 | 5 | 919552 | 919552 | 110% | 166% | 76% | 76% |
| cases/shrinkarray.rb | 1.932001 | 9 | 200000 | 200000 | 2.122906 | 9 | 100352 | 77824 | 109% | 100% | 50% | 38% |
| cases/ostruct_read_yaml.rb | 1.142735 | 20 | 2374400 | 2374400 | 1.114133 | 17 | 3178496 | 3178496 | 97% | 85% | 133% | 133% |
| cases/hashes_read.rb | 0.0522 | 3 | 1208000 | 1208000 | 0.052393 | 4 | 876544 | 876544 | 100% | 133% | 72% | 72% |
| 102% | 110% | 75% | 55% | |||||||||
feedback
send perf feedback to lloyd at trickyco.de
[...] nature of the heap mechanism in ruby, so perhaps we need to retry this with some of them gc patches I wrote, or with [...]
bqCBVE hfN2cnPosa82GhrpCayYr