Issue #5248 has been updated by Shota Fukumori.

Status changed from Open to Assigned


----------------------------------------
Feature #5248: Faster PStore
https://bugs.ruby-lang.org/issues/5248

Author: Masaki Matsushita
Status: Assigned
Priority: Normal
Assignee: Hiroshi Nakamura
Category: lib
Target version: 2.0.0


=begin
Hellow.

I wrote a patch to make PStore more faster.
What I did as follows:

:deferred check sum calculation

  PStore judges whether it should modify database file or not by 2 steps.
  First, it compares data size between database file and marshal data to write.
  If both size are different, it writes data to database file.
  Second, if both sizes are same, it compares Digest::MD5.digest of both still more.

  However, PStore calculates a check sum of data to write before size comparison.
  If PStore can judge it should modify database file in size comparison, this calculation will be useless.
  Consequently, I modified PStore to calculate a check sum after size comparison.

:check-sum calculation by String#sum

  As stated above, Present PStore calculates checksum of database file by Digest::MD5.digest to judge whether it should modify file or not.
  However, Digest::MD5.digest is cryptographic hash function and I think it is too strong to use as a mere check sum.
  Therefore, I modified PStore to use not Digest::MD5.digest but String#sum.

:deferred File#truncate

  PStore puts back file pointer to the head and truncates file size to zero before writing to database file as below.

    file.rewind
    file.truncate(0)
    file.write(data)
    (pstore.rb at line 486~488)

  However, truncation by File#truncate is slow and it is the bottleneck of PStore.
  I modified it as below.

    file.rewind
    file.write(data)
    file.truncate(data.size)

  It only puts back file pointer before write. Truncation is done after writing.
  In this way, size needs to be truncate will be minimum and it makes PStore faster.

:performance

  I benchmarked PStore as below:

    require 'pstore'
    
    p = PStore.new("foo")
    p.transaction { p["hoge"] = "hoge" * ARGV.first.to_i }
    
    10000.times do
      p.transaction { p["hoge"] += "hoge" }
    end
    
  Present PStore:

    % time ruby pstore_bench.rb 1000
    ruby pstore_bench.rb 1000  2.94s user 2.43s system 69% cpu 7.723 total
    % time ruby pstore_bench.rb 10000
    ruby pstore_bench.rb 10000  5.37s user 2.99s system 70% cpu 11.810 total
    % time ruby pstore_bench.rb 100000
    ruby pstore_bench.rb 100000  31.98s user 11.09s system 69% cpu 1:02.15 total

  New PStore:

    % time ruby pstore_bench.rb 1000
    ruby pstore_bench.rb 1000  1.67s user 0.44s system 99% cpu 2.119 total
    % time ruby pstore_bench.rb 10000
    ruby pstore_bench.rb 10000  3.24s user 0.63s system 99% cpu 3.876 total
    % time ruby pstore_bench.rb 100000
    ruby pstore_bench.rb 100000  14.29s user 3.13s system 100% cpu 17.416 total

As a result, new PStore is faster.
It can be said that new PStore is the faster, the bigger database file is.
I attached a patch. PStore applied the patch passes test-all.
=end


-- 
http://redmine.ruby-lang.org