On Sep 4, 2006, at 8:06 PM, Rick DeNatale wrote: > So, I've corrected the benchmark. I've also added another method > using String#split as Eric suggested. > > Here's the code being benchmarked: > rick@frodo:~/rubyscripts$ cat stringsplit.rb Putting the code here involves an extra method call. While they are all wrapped in a method call equally, doing less is always better. > class String > [methods] > end > > Note that I made tested three different "pre-compiled" regex's one > with a Regex.new('.'), one with Regex.new(/./), and one a constant > with the literal regex /./. You don't need three different benchmarks for this. Its easy to determine that these are the same with irb. > Now here's the benchmark: > rick@frodo:~/rubyscripts$ cat benchstringsplit.rb > require 'benchmark' > include Benchmark > load 'stringsplit.rb' > > iters = 100 100 iterations is never enough. GC behavior, other processes waking up, etc. will all cause fluctuations in the benchmark. Use 100_000 or 1_000_000. > [benchmark code] Here's a more-meaningful version of yours: $ cat stringsbench.rb require 'benchmark' N = 100_000 RE = /./ str = "abcdefghijklmnopqrstuvwxyz" * 5 Benchmark.bmbm do | x | x.report 'empty' do N.times do end end x.report 'str' do N.times do str end end x.report 'unpack' do N.times do str.unpack('a' * str.length) end end x.report 'scan RE' do N.times do str.scan RE end end x.report 'scan /./' do N.times do str.scan /./ end end x.report 'split //' do N.times do str.split // end end end $ ruby stringsbench.rb Rehearsal -------------------------------------------- empty 0.030000 0.000000 0.030000 ( 0.088456) str 0.050000 0.000000 0.050000 ( 0.064515) unpack 14.370000 0.210000 14.580000 ( 20.264274) scan RE 24.240000 0.380000 24.620000 ( 36.296017) scan /./ 24.100000 0.310000 24.410000 ( 32.661832) split // 29.660000 0.360000 30.020000 ( 40.596177) ---------------------------------- total: 93.710000sec user system total real empty 0.030000 0.000000 0.030000 ( 0.045199) str 0.050000 0.010000 0.060000 ( 0.065739) unpack 14.500000 0.190000 14.690000 ( 19.813677) scan RE 23.850000 0.350000 24.200000 ( 33.028376) scan /./ 23.910000 0.320000 24.230000 ( 35.219831) split // 29.720000 0.470000 30.190000 ( 42.712222) with 'abc...xyz' * 25: Rehearsal -------------------------------------------- empty 0.050000 0.000000 0.050000 ( 0.217969) str 0.050000 0.000000 0.050000 ( 0.087123) unpack 65.370000 0.710000 66.080000 ( 80.341765) scan RE 107.840000 1.030000 108.870000 (129.723449) scan /./ 110.320000 1.250000 111.570000 (142.540348) split // 135.220000 1.330000 136.550000 (164.418453) --------------------------------- total: 423.170000sec user system total real empty 0.030000 0.000000 0.030000 ( 0.040596) str 0.050000 0.000000 0.050000 ( 0.066506) unpack 62.130000 0.590000 62.720000 ( 70.183396) scan RE 107.730000 0.940000 108.670000 (125.308283) scan /./ 107.550000 0.950000 108.500000 (127.365719) split // 133.700000 1.020000 134.720000 (150.798611) > So, at least from this benchmark it doesn't seem that in-line literal > regular expressions are faster than pre-compiled ones. inline regular expressions are no less "pre-compiled" than regular expressions in a variable or constant. $ ruby -e '2.times do puts /./.object_id end' 938970 938970 ('=~ /./' is faster than '=~ var' is faster than 'match anything', but for other reasons) If you want to test inline vs "pre-compiled" regular expressions you need to throw away all the parts that are the same and focus on what is different. In your benchmark it was what was on the right hand side of String#scan. Since the string was the same and the scan was the same, just throw those away. You end up with a benchmark like this: $ cat vs.rb require 'benchmark' N = 100_000_000 RE = /./ re = /./ $re = /./ @re = /./ @@re = /./ Benchmark.bmbm do |bm| bm.report 'empty' do N.times do end end bm.report 'lit' do N.times do /./ end end bm.report 'local' do N.times do re end end bm.report 'global' do N.times do $re end end bm.report 'instance' do N.times do @re end end bm.report 'class' do N.times do @@re end end bm.report 'constant' do N.times do RE end end end $ ruby vs.rb Rehearsal -------------------------------------------- empty 27.930000 0.120000 28.050000 ( 31.550709) lit 48.240000 0.320000 48.560000 ( 57.422011) local 48.820000 0.290000 49.110000 ( 60.650110) global 49.920000 0.420000 50.340000 ( 62.156780) instance 55.740000 0.180000 55.920000 ( 61.027820) class 57.800000 0.190000 57.990000 ( 63.733099) constant 59.240000 0.330000 59.570000 (119.118487) --------------------------------- total: 349.540000sec user system total real empty 27.820000 0.160000 27.980000 ( 58.584698) lit 48.170000 0.170000 48.340000 ( 59.557450) local 48.720000 0.170000 48.890000 ( 53.775692) global 49.830000 0.180000 50.010000 ( 55.743586) instance 55.880000 0.370000 56.250000 ( 72.011443) class 57.910000 0.390000 58.300000 ( 67.037158) constant 59.230000 0.410000 59.640000 ( 77.993887) So literal regular expressions are "faster" by about 56% (after discounting loop overhead) (when you're performing 100 million retrievals only) (actual speedup in real code will probably be swallowed elsewhere or completely irrelevant). This follows from reading eval.c. Constant lookup involves a bunch of C function calls, but literal lookup just returns an Object stored in the parse tree. > In fact they never came in first, although PCR3 which was a > constant set to a > literal regex did win 4 times. In fact, when you adjust the stringsbench.rb you'll see /./ winning over RE. Set N to 10_000_000 (or more) and str to "abcde" (or just N high enough). > Is this significant? Who knows, and with a new RegExp engine coming > the numbers will be different in future. The difference was probably due to processes waking up, garbage collection and similar unevenly distributed events. With more iterations and more focused benchmarks you'll get better results. But ultimately, these types of microbenchmarks are not very useful. Yes, you will get a speedup using /./ over RE, but will your program really run long enough where it matters to make a difference? I bet not. > So as usual the right approach is test, profile to find out what needs > improvement, and benchmark. Be careful with your benchmarks, they are most useful when they are as small and simple as possible. Be sure to throw away all the irrelevant parts. Be careful with your benchmarks, they are most useless when they are as small and simple as possible. Be sure to understand how little speedup you'll get. -- Eric Hodel - drbrain / segment7.net - http://blog.segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com