------------lUJwCF93a8UQUMWapAZpOz Content-Type: text/plain; format=flowed; delsp=yes; charset=utf-8 Content-Transfer-Encoding: 8bit Hello, My colleagues and I have discovered a potential problem with the implementation of the CGI library's read_multipart method. The current CGI implementation will create a temporary file for many of the multipart data entries if the posted content length is greater than 10 kilobytes. I would like to ask what is the intended goal of the implementation? Is the goal to make temporary files for each data entry that is more than 10 kilobytes and keep entries that are less in StringIO objects? The reason I ask is that the current implementation works in a different way. It will create temporary files for each data entry until the remaining content data stream is less than 10 kilobytes. Then it will use StringIO for the remaining entries. This implementation turns out to have disastrous consequences for some of my applications. We have some situations where we send images and many select options in a post. This ends up creating hundreds and sometimes thousands of Tempfiles, which causes our servers to become unresponsive. I have modified read_multipart to only create TempFiles for each data entry greater than 10 kilobytes. With this modification our servers are responsive and show no other problems with my change. I have made a few other changes to the implementation to use sub! method calls to hopefully remove unneeded duplication of large buffers. I dug through the CVS history and it appears it was originally using such a sub! call, but later when to "buf uf.sub" calls, is there a reason why? If so I will revert back to "buf uf.sub". Ignoring a few minor style changes in my implementation and or my implementation all together, the main point of discussion I would like to start is if the current 10 kilobyte implementation is correct of if the intended goal of my modification is correct? If my modification is not too offensive I can integrate it with cgi.rb and post a patch. Best Regards, Zev Blut ------------lUJwCF93a8UQUMWapAZpOz Content-Disposition: attachment; filename=CGIFIX.rb Content-Type: application/octet-stream; name=CGIFIX.rb Content-Transfer-Encoding: Base64 IyBUaGlzIGlzIGEgcXVpY2sgZml4IHRvIGNoYW5nZSBDR0kncyByZWFkX211dGlw YXJ0IG1ldGhvZCB0byBub3QKIyBjcmVhdGUgbG90cyBvZiBzbWFsbCBUZW1wZmls ZXMuICBJdCB3aWxsIG5vdyBvbmx5IGNyZWF0ZSBUZW1wZmlsZXMKIyBmb3IgZGF0 YSBlbnRyaWVzIHRoYXQgYXJlIGxhcmdlciB0aGFuIDEwS0IuCgpjbGFzcyBDR0kK CiAgbW9kdWxlIFF1ZXJ5RXh0ZW5zaW9uCgogICAgZGVmIHJlYWRfbXVsdGlwYXJ0 KGJvdW5kYXJ5LCBjb250ZW50X2xlbmd0aCkKICAgICAgcGFyYW1zID0gSGFzaC5u ZXcoKSB7IHxoLGt8IGhba10gPSBBcnJheS5uZXcgfQogICAgICBib3VuZGFyeSA9 ICItLSIgKyBib3VuZGFyeQogICAgICBidWYgPSAiIgogICAgICBidWZzaXplID0g MTAyNDAgIyAxMEtCCgogICAgICAjIHN0YXJ0IG11bHRpcGFydC9mb3JtLWRhdGEK ICAgICAgc3RkaW5wdXQuYmlubW9kZSBpZiBkZWZpbmVkPyBzdGRpbnB1dC5iaW5t b2RlCiAgICAgIGJvdW5kYXJ5X3NpemUgPSBib3VuZGFyeS5zaXplICsgRU9MLnNp emUKICAgICAgY29udGVudF9sZW5ndGggLT0gYm91bmRhcnlfc2l6ZQoKICAgICAg c3RhdHVzID0gc3RkaW5wdXQucmVhZChib3VuZGFyeV9zaXplKQogICAgICBpZiBu aWwgPT0gc3RhdHVzCiAgICAgICAgcmFpc2UgRU9GRXJyb3IsICJubyBjb250ZW50 IGJvZHkiCiAgICAgIGVsc2lmIGJvdW5kYXJ5ICsgRU9MICE9IHN0YXR1cwogICAg ICAgIHJhaXNlIEVPRkVycm9yLCAiYmFkIGNvbnRlbnQgYm9keSIKICAgICAgZW5k CgogICAgICBsb29wIGRvCiAgICAgICAgaGVhZCA9IG5pbAogICAgICAgIGJvZHkg PSBNb3JwaGluZ0JvZHkubmV3CgogICAgICAgIHVudGlsIGhlYWQgYW5kIC8je2Jv dW5kYXJ5fSg/OiN7RU9MfXwtLSkvbi5tYXRjaChidWYpCgogICAgICAgICAgaWYg KG5vdCBoZWFkKSBhbmQgLyN7RU9MfSN7RU9MfS9uLm1hdGNoKGJ1ZikKICAgICAg ICAgICAgYnVmLnN1YiEoL1xBKCg/Oi58XG4pKj8je0VPTH0pI3tFT0x9L24pIGRv CiAgICAgICAgICAgICAgaGVhZCA9ICQxLmR1cAogICAgICAgICAgICAgICIiCiAg ICAgICAgICAgIGVuZAogICAgICAgICAgICBuZXh0CiAgICAgICAgICBlbmQKCiAg ICAgICAgICBpZiBoZWFkIGFuZCAoIChFT0wgKyBib3VuZGFyeSArIEVPTCkuc2l6 ZSA8IGJ1Zi5zaXplICkKICAgICAgICAgICAgYm9keS5wcmludCBidWZbMCAuLi4g KGJ1Zi5zaXplIC0gKEVPTCArIGJvdW5kYXJ5ICsgRU9MKS5zaXplKV0KICAgICAg ICAgICAgYnVmWzAgLi4uIChidWYuc2l6ZSAtIChFT0wgKyBib3VuZGFyeSArIEVP TCkuc2l6ZSldID0gIiIKICAgICAgICAgIGVuZAoKICAgICAgICAgICMgc2hvdWxk IGJlIGMgPSBpZiwgYnV0IGNoYW5nZWQgZm9yIGJyb2tlbiBlbWFjcyBpbmRlbnQK ICAgICAgICAgIGlmIGJ1ZnNpemUgPCBjb250ZW50X2xlbmd0aAogICAgICAgICAg ICBjID0gc3RkaW5wdXQucmVhZChidWZzaXplKQogICAgICAgICAgZWxzZQogICAg ICAgICAgICBjID0gc3RkaW5wdXQucmVhZChjb250ZW50X2xlbmd0aCkKICAgICAg ICAgIGVuZAoKICAgICAgICAgIGlmIGMubmlsPwogICAgICAgICAgICByYWlzZSBF T0ZFcnJvciwgImJhZCBjb250ZW50IGJvZHkiCiAgICAgICAgICBlbmQKICAgICAg ICAgIGJ1Zi5jb25jYXQoYykKICAgICAgICAgIGNvbnRlbnRfbGVuZ3RoIC09IGMu c2l6ZQogICAgICAgIGVuZAoKICAgICAgICAjIGNoYW5nZSB0byAhIHRvIHJlbW92 ZSBleGNlc3NpdmUgb2JqZWN0IGNyZWF0aW9uCiAgICAgICAgYnVmLnN1YiEoL1xB KCg/Oi58XG4pKj8pKD86W1xyXG5dezEsMn0pPyN7Ym91bmRhcnl9KFtcclxuXXsx LDJ9fC0tKS9uKSBkbwogICAgICAgICAgYm9keS5wcmludCAkMQogICAgICAgICAg aWYgIi0tIiA9PSAkMgogICAgICAgICAgICBjb250ZW50X2xlbmd0aCA9IC0xCiAg ICAgICAgICBlbmQKICAgICAgICAgICIiCiAgICAgICAgZW5kCgogICAgICAgIGJv ZHkgPSBib2R5LmV4dHJhY3QKICAgICAgICBib2R5LnJld2luZAoKICAgICAgICAv Q29udGVudC1EaXNwb3NpdGlvbjouKiBmaWxlbmFtZT0iPyhbXlwiO10qKSI/L25p Lm1hdGNoKGhlYWQpCiAgICAgICAgZmlsZW5hbWUgPSAoJDEgb3IgIiIpCiAgICAg ICAgaWYgL01hYy9uaS5tYXRjaChlbnZfdGFibGVbJ0hUVFBfVVNFUl9BR0VOVCdd KSBhbmQKICAgICAgICAgICAgL01vemlsbGEvbmkubWF0Y2goZW52X3RhYmxlWydI VFRQX1VTRVJfQUdFTlQnXSkgYW5kCiAgICAgICAgICAgIChub3QgL01TSUUvbmku bWF0Y2goZW52X3RhYmxlWydIVFRQX1VTRVJfQUdFTlQnXSkpCiAgICAgICAgICBm aWxlbmFtZSA9IENHSTo6dW5lc2NhcGUoZmlsZW5hbWUpCiAgICAgICAgZW5kCgog ICAgICAgIC9Db250ZW50LVR5cGU6ICguKikvbmkubWF0Y2goaGVhZCkKICAgICAg ICBjb250ZW50X3R5cGUgPSAoJDEgb3IgIiIpCgogICAgICAgIChjbGFzcyA8PCBi b2R5OyBzZWxmOyBlbmQpLmNsYXNzX2V2YWwgZG8KICAgICAgICAgIGFsaWFzIGxv Y2FsX3BhdGggcGF0aAogICAgICAgICAgZGVmaW5lX21ldGhvZCg6b3JpZ2luYWxf ZmlsZW5hbWUpIHtmaWxlbmFtZS5kdXAudGFpbnR9CiAgICAgICAgICBkZWZpbmVf bWV0aG9kKDpjb250ZW50X3R5cGUpIHtjb250ZW50X3R5cGUuZHVwLnRhaW50fQog ICAgICAgIGVuZAoKICAgICAgICAvQ29udGVudC1EaXNwb3NpdGlvbjouKiBuYW1l PSI/KFteXCI7XSopIj8vbmkubWF0Y2goaGVhZCkKICAgICAgICBuYW1lID0gJDEu ZHVwCgogICAgICAgIHBhcmFtc1tuYW1lXS5wdXNoKGJvZHkpCiAgICAgICAgYnJl YWsgaWYgYnVmLnNpemUgPT0gMAogICAgICAgIGJyZWFrIGlmIGNvbnRlbnRfbGVu Z3RoID09IC0xICMgcmVtb3ZlZCA9PT0gPwogICAgICBlbmQKCiAgICAgIHBhcmFt cwogICAgZW5kICMgcmVhZF9tdWx0aXBhcnQKCiAgICByZXF1aXJlICJzdHJpbmdp byIKICAgIHJlcXVpcmUgInRlbXBmaWxlIgoKICAgICMgQSB3cmFwcGVyIGNsYXNz IHRvIHVzZSBhIFN0cmluZ0lPIG9iamVjdCBhcyB0aGUgYm9keSBhbmQgc3dpdGNo CiAgICAjIHRvIGEgVGVtcEZpbGUgd2hlbiB0aGUgcGFzc2VkIHRocmVzaG9sZCBp cyBwYXNzZWQuCiAgICBjbGFzcyBNb3JwaGluZ0JvZHkKCiAgICAgIGRlZiBpbml0 aWFsaXplKG1vcnBoX3RocmVzaG9sZCA9IDEwMjQwKQogICAgICAgIEB0aHJlc2hv bGQgPSBtb3JwaF90aHJlc2hvbGQKICAgICAgICBAYm9keSA9IFN0cmluZ0lPLm5l dwogICAgICAgIEBjdXJfc2l6ZSA9IDAKICAgICAgICBAbW9ycGhfY2hlY2sgPSB0 cnVlCgogICAgICAgIEBib2R5LmJpbm1vZGUgaWYgZGVmaW5lZD8gQGJvZHkuYmlu bW9kZQogICAgICBlbmQKCiAgICAgIGRlZiBwcmludChkYXRhKQogICAgICAgIGlm IEBtb3JwaF9jaGVjayAmJiAoQGN1cl9zaXplICsgZGF0YS5zaXplID4gQHRocmVz aG9sZCkKICAgICAgICAgIGNvbnZlcnRfYm9keQogICAgICAgIGVuZAogICAgICAg IEBib2R5LnByaW50IGRhdGEKICAgICAgZW5kCgogICAgICAjIHJldHVybnMgdGhl IHRydWUgYm9keSBvYmplY3QuCiAgICAgIGRlZiBleHRyYWN0CiAgICAgICAgQGJv ZHkKICAgICAgZW5kCgogICAgICBwcml2YXRlCiAgICAgIGRlZiBjb252ZXJ0X2Jv ZHkKICAgICAgICBuZXdfYm9keSA9IFRlbXBGaWxlLm5ldygiQ0dJIikKICAgICAg ICBuZXdfYm9keS5iaW5tb2RlIGlmIGRlZmluZWQ/IG5ld19ib2R5LmJpbm1vZGUK CiAgICAgICAgQGJvZHkucmV3aW5kCiAgICAgICAgbmV3X2JvZHkucHJpbnQgQGJv ZHkucmVhZAogICAgICAgIEBib2R5ID0gbmV3X2JvZHkKICAgICAgICBAbW9ycGhf Y2hlY2sgPSBmYWxzZQogICAgICBlbmQKICAgIGVuZAoKICBlbmQKCmVuZAo-----------lUJwCF93a8UQUMWapAZpOz--