--mimepart_4d3f6ef1a88ea_138658e14ee555f
Content-Type: text/plain
Content-Transfer-Encoding: Quoted-printable
Content-Disposition: inline

Bug #4325: [ext/openssl] Encoding of subclasses fails when it shouldn't
http://redmine.ruby-lang.org/issues/show/4325

Author: Martin Bosslet
Status: Open, Priority: Normal
Category: ext, Target version: 1.9.3
ruby -v: trunk

While skimming through ossl_asn1.c I noticed that my patch for infinite length encoding
causes problems when encoding subclasses of OpenSSL::ASN1::Sequence or OpenSSL::ASN1::Set
with infinite length.

E.g. the following fails in trunk:

require 'openssl'

sub = Class.new(OpenSSL::ASN1::Sequence)
instance = sub.new([OpenSSL::ASN1::EndOfContent.new])
instance.infinite_length = true
puts instance.to_der

=> test.rb:10:in `to_der': invalid constructed encoding (OpenSSL::ASN1::ASN1Error)
	from test.rb:10:in `<main>'

This can be fixed with the appended code that checks for subclass relationship instead
of comparing the class directly with Set or Sequence.

Regards,
Martin


----------------------------------------
http://redmine.ruby-lang.org

--mimepart_4d3f6ef1a88ea_138658e14ee555f
Content-Type: text/x-patch; name=fix_cons_encode_inf.diff
Content-Transfer-Encoding: Base64
Content-Disposition: attachment; filename=fix_cons_encode_inf.diff

ClByb3BlcnR5IGNoYW5nZXMgb246IHJ1YnkvZXh0L29wZW5zc2wKX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fXwpNb2RpZmllZDogc3ZuOmlnbm9yZQogICAtIEdO
VW1ha2VmaWxlCk1ha2VmaWxlCmRlcApleHRjb25mLmgKbWttZi5sb2cKb3Bl
bnNzbC5hCmNvbmZ0ZXN0LmRTWU0KCiAgICsgR05VbWFrZWZpbGUKTWFrZWZp
bGUKZGVwCmV4dGNvbmYuaApta21mLmxvZwpvcGVuc3NsLmEKY29uZnRlc3Qu
ZFNZTQpvcGVuc3NsLnNvCgoKSW5kZXg6IHJ1YnkvZXh0L29wZW5zc2wvb3Nz
bF9hc24xLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gcnVieS9leHQv
b3BlbnNzbC9vc3NsX2FzbjEuYwkocmV2aXNpb24gMzA2NDApCisrKyBydWJ5
L2V4dC9vcGVuc3NsL29zc2xfYXNuMS5jCSh3b3JraW5nIGNvcHkpCkBAIC0x
MDE3LDE4ICsxMDE3LDIyIEBACiAgICAgaW50IGZvdW5kX3ByaW0gPSAwOwog
ICAgIGxvbmcgc2VxX2xlbiwgbGVuZ3RoOwogICAgIHVuc2lnbmVkIGNoYXIg
KnA7Ci0gICAgVkFMVUUgdmFsdWUsIHN0ciwgaW5mX2xlbmd0aCwgYXJ5LCBl
eGFtcGxlOworICAgIFZBTFVFIHZhbHVlLCBzdHIsIGluZl9sZW5ndGgsIGFy
eSwgZXhhbXBsZSA9IFFuaWw7CiAKICAgICB0biA9IE5VTTJJTlQob3NzbF9h
c24xX2dldF90YWcoc2VsZikpOwogICAgIHRjID0gb3NzbF9hc24xX3RhZ19j
bGFzcyhzZWxmKTsKICAgICBpbmZfbGVuZ3RoID0gb3NzbF9hc24xX2dldF9p
bmZpbml0ZV9sZW5ndGgoc2VsZik7CiAgICAgaWYgKGluZl9sZW5ndGggPT0g
UXRydWUpIHsKICAgICAgICAgY29uc3RydWN0ZWQgPSAyOwotICAgICAgICBp
ZiAoQ0xBU1NfT0Yoc2VsZikgPT0gY0FTTjFTZXF1ZW5jZSB8fAotICAgICAg
ICAgICAgQ0xBU1NfT0Yoc2VsZikgPT0gY0FTTjFTZXQpIHsKKwlpZiAocmJf
b2JqX2lzX2tpbmRfb2Yoc2VsZiwgY0FTTjFTZXF1ZW5jZSkgfHwKKyAgICAg
ICAgICAgIHJiX29ial9pc19raW5kX29mKHNlbGYsIGNBU04xU2V0KSl7CiAg
ICAgICAgICAgICB0YWcgPSBvc3NsX2FzbjFfZGVmYXVsdF90YWcoc2VsZik7
CiAgICAgICAgIH0KLSAgICAgICAgZWxzZSB7IC8qQklUX1NUUklORyBPUiBP
Q1RFVF9TVFJJTkcqLworCWVsc2UgeyAvKm11c3QgYmUgYSBjb25zdHJ1Y3Rp
dmUgZW5jb2Rpbmcgb2YgYSBwcmltaXRpdmUgdmFsdWUqLworCSAgICBpZiAo
IXJiX29ial9pc19raW5kX29mKHNlbGYsIGNBU04xQ29uc3RydWN0aXZlKSl7
CisJCW9zc2xfcmFpc2UoZUFTTjFFcnJvciwgImludmFsaWQgY29uc3RydWN0
ZWQgZW5jb2RpbmciKTsKKyAgICAgICAgICAgICAgICByZXR1cm4gUW5pbDsg
LyogZHVtbXkgKi8KKyAgICAgICAgICAgIH0KICAgICAgICAgICAgIGFyeSA9
IG9zc2xfYXNuMV9nZXRfdmFsdWUoc2VsZik7CiAgICAgICAgICAgICAvKiBS
ZWN1cnNpdmVseSBkZXNjZW5kIHVudGlsIGEgcHJpbWl0aXZlIHZhbHVlIGlz
IGZvdW5kLgogICAgICAgICAgICAgICAgVGhlIG92ZXJhbGwgdmFsdWUgb2Yg
dGhlIGVudGlyZSBjb25zdHJ1Y3RlZCBlbmNvZGluZwpJbmRleDogcnVieS90
ZXN0L29wZW5zc2wvdGVzdF9hc24xLnJiCj09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT0KLS0tIHJ1YnkvdGVzdC9vcGVuc3NsL3Rlc3RfYXNuMS5yYgkocmV2aXNp
b24gMzA2NDApCisrKyBydWJ5L3Rlc3Qvb3BlbnNzbC90ZXN0X2FzbjEucmIJ
KHdvcmtpbmcgY29weSkKQEAgLTQzMCw0ICs0MzAsMTggQEAKICAgICBlbmQK
ICAgZW5kCiAgIAorICBkZWYgdGVzdF9lbmNvZGVfc3ViY2xhc3Nfc2VxdWVu
Y2UKKyAgICBzdWIgPSBDbGFzcy5uZXcoT3BlblNTTDo6QVNOMTo6U2VxdWVu
Y2UpCisgICAgaW5zdGFuY2UgPSBzdWIubmV3KFtPcGVuU1NMOjpBU04xOjpF
bmRPZkNvbnRlbnQubmV3XSkKKyAgICBpbnN0YW5jZS5pbmZpbml0ZV9sZW5n
dGggPSB0cnVlCisgICAgcHV0cyBpbnN0YW5jZS50b19kZXIKKyAgZW5kCisg
IAorICBkZWYgdGVzdF9lbmNvZGVfc3ViY2xhc3Nfc2V0CisgICAgc3ViID0g
Q2xhc3MubmV3KE9wZW5TU0w6OkFTTjE6OlNldCkKKyAgICBpbnN0YW5jZSA9
IHN1Yi5uZXcoW09wZW5TU0w6OkFTTjE6OkVuZE9mQ29udGVudC5uZXddKQor
ICAgIGluc3RhbmNlLmluZmluaXRlX2xlbmd0aCA9IHRydWUKKyAgICBwdXRz
IGluc3RhbmNlLnRvX2RlcgorICBlbmQKKyAgCiBlbmQgaWYgZGVmaW5lZD8o
T3BlblNTTCkK

--mimepart_4d3f6ef1a88ea_138658e14ee555f--