------------GbwtIOsuhVxnZ0s04hHuwa
Content-Type: text/plain; format=flowed; delsp=yes; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
Attached is a patch that implements "Encoding::default_internal". The aim
of this patch is to allow a Ruby programmer to specify a default encoding
that all IO (and I hope other methods) will return strings in, so that you
can deal with the strings without worrying (too much) about encoding
compatibility.
In doing this patch I have attempted to take everyone's input into account.
I have spent quite a bit of time on this (mainly thinking about the
features that will make it useful for everyone), so I hope that it is
accepted, even if it is a bit late!
A summary of what this patch does:
1) Extends the "-E" command line flag to accept either "ext_enc",
"ext_enc:int_enc" or ":int_enc". If the form "ext_enc:int_enc" is used,
the value of BOTH default_external & default_internal is set. If the form
":int_enc" is used, only the default_internal is set - the
default_external is left at it's default value. Similarly if the form
"ext_enc" is used - default_internal is left at it's default value.
2) Extended the "magic comment" feature to also look for
"internal_encoding: XXXX". If this is found in the MAIN ruby file only,
both the source encoding AND the default_internal are set to the given
encoding. If found in an included source file (eg: library), it is treated
the same way as "encoding:" and sets the source encoding only. (You
shouldn't use it in a library anyhow).
3) The above 2 features give the Ruby programmer 2 ways of specifying the
value of default_internal in their main program:
- By using "internal_encoding: XXX" magic comment, or
- By using -E:XXX in the ruby "shebang" on the first line (note the ":"
before the XXX to indicate "internal encoding").
The second form allows the programmer to specify a different value for
"default_internal" than the source encoding (not sure if that is
particularly useful).
Use of both of these is ignored when not in the main program, so that
"default_internal" can only be set once and only under the control of the
programmer or user.
Note: If the default_internal is specified on the ruby command line, it
overrides both of these.
4) If for some misguided reason default_internal is set to US-ASCII, it
actually sets itself to UTF-8, as this is much more likely to cope better.
5) The new method Encoding#default_internal works just like
Encoding#default_external - it is read-only. If it is not set, it's value
is nil. The is no "default_internal method.
6) When setting the encodings of an IO, if no internal encoding is
specified, the default_internal value will be used if it is set. If it is
not set, there is no transcoding.
7) There is one exception to the use of "default_internal" when opening a
file with an external encoding of ASCII-8BIT (aka BINARY). In this case
"default_internal" is ignored, and data returned untranscoded because the
user probably wants to do byte or bit processing rather than character.
8) The "int_enc" passed in the file mode or to IO#set_encoding may also be
"-". This will force the internal encoding to nil for the file, so no
transcoding is done on input.
For example: open(file, "r:EUC-JP:-") or f.set_encoding("EUC-JP", "-").
This gives the programmer the ability to prevent transcoding even if
"default_internal" is set.
Notes:
- When default_internal is not set Ruby works exactly as it does now in
1.9.0
- "Unicode whiners" (not my expression!) should probably set
"default_internal" to UTF-8.
- "default_internal" does *not* guarantee that all strings in your program
will be in that encoding. What it does (currently) is work on IO when
reading files or sockets etc. You can still use String#encode,
String#force_encoding etc to change them. It would be nice to actually
have a way of *forcing* all strings (and Regexps) to be a particular
encoding in order to guarantee no "encoding compatibility" problems.
However this is quite a bit more complicated to implement. So when writing
libraries & modules, you may still have to check the encoding of strings
passed to your methods.
- Using the "internal_encoding:" magic comment in your main program is the
preferred way of setting default_internal. This is because both your
source encoding and data read from files will be the same encoding, so at
least your program's string literals should match too.
- there are probably other libraries that should be patched to support
"default_internal" to ensure that strings are returned to the user in that
encoding. However ones like OpenURI *should* just work, because of the way
"default_internal" is supported transparently in the IO class.
- [I have not implemented this, because it would probably be rejected.
However I'll say it because I still think it is a good idea] I think if a
file is open with "b" mode, it should default to ASCII-8BIT, no
transcoding. This would probably get rid of the need for "bin_read" too.
Implementation notes:
- encoding.c/encoding.h - added functions to get & set "default_internal"
- parse.y - added code to support the "internal_encoding" magic comment
- ruby.c - added code to extend -E/--encoding option
- io.c - added code to support default_internal. Note that I almost
completely rewrote parse_mode_enc(). The other main mod is a new function
rb_io_ext_int_to_encs() which converts the external & internal encodings
to values for "enc" & "enc2" in convconfig_t. It is shame that "enc" &
"enc2" don't map directly to "internal" and "external" - it would have
made the code much simpler!
- Testing: not extensively tested. Passes "make test". Some "make
test-all" tests fail (esp rdoc), but I think that they may be due to other
issues (not certain).
I'm sure you'll tell me if I have broken anything!
Cheers
Mike
------------GbwtIOsuhVxnZ0s04hHuwa
Content-Disposition: attachment; filename=default_internal.pat
Content-Type: application/octet-stream; name=default_internal.pat
Content-Transfer-Encoding: Base64
SW5kZXg6IGVuY29kaW5nLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gZW5jb2Rp
bmcuYwkocmV2aXNpb24gMTk1NjEpCisrKyBlbmNvZGluZy5jCSh3b3JraW5nIGNv
cHkpCkBAIC0xMDI3LDggKzEwMjcsNTcgQEAKICAgICBkZWZhdWx0X2V4dGVybmFs
ID0gMDsKIH0KIAorLyogLTIgPT4gbm90IHlldCBzZXQsIC0xID0+IG5pbCAqLwor
c3RhdGljIGludCBkZWZhdWx0X2ludGVybmFsX2luZGV4ID0gLTI7CitzdGF0aWMg
cmJfZW5jb2RpbmcgKmRlZmF1bHRfaW50ZXJuYWw7CisKK3JiX2VuY29kaW5nICoK
K3JiX2RlZmF1bHRfaW50ZXJuYWxfZW5jb2Rpbmcodm9pZCkKK3sKKyAgICBpZiAo
IWRlZmF1bHRfaW50ZXJuYWwgJiYgZGVmYXVsdF9pbnRlcm5hbF9pbmRleCA+PSAw
KSB7CisJZGVmYXVsdF9pbnRlcm5hbCA9IHJiX2VuY19mcm9tX2luZGV4KGRlZmF1
bHRfaW50ZXJuYWxfaW5kZXgpOworICAgIH0KKyAgICByZXR1cm4gZGVmYXVsdF9p
bnRlcm5hbDsKK30KKworVkFMVUUKK3JiX2VuY19kZWZhdWx0X2ludGVybmFsKHZv
aWQpCit7CisgICAgLyogTm90ZTogVGhlc2UgZnVuY3Rpb25zIGNvcGUgd2l0aCBk
ZWZhdWx0X2ludGVybmFsIG5vdCBiZWluZyBzZXQgKi8KKyAgICByZXR1cm4gcmJf
ZW5jX2Zyb21fZW5jb2RpbmcocmJfZGVmYXVsdF9pbnRlcm5hbF9lbmNvZGluZygp
KTsKK30KKwogLyoKICAqIGNhbGwtc2VxOgorICogICBFbmNvZGluZy5kZWZhdWx0
X2ludGVybmFsID0+IGVuYworICoKKyAqIFJldHVybnMgZGVmYXVsdCBpbnRlcm5h
bCBlbmNvZGluZy4KKyAqCisgKiBJdCBpcyBpbml0aWFsaXplZCBieSB0aGUgc291
cmNlIGludGVybmFsX2VuY29kaW5nIG9yIC1FIG9wdGlvbiwKKyAqIGFuZCBjYW4n
dCBiZSBtb2RpZmllZCBhZnRlciB0aGF0LgorICovCitzdGF0aWMgVkFMVUUKK2dl
dF9kZWZhdWx0X2ludGVybmFsKFZBTFVFIGtsYXNzKQoreworICAgIHJldHVybiBy
Yl9lbmNfZGVmYXVsdF9pbnRlcm5hbCgpOworfQorCit2b2lkCityYl9lbmNfc2V0
X2RlZmF1bHRfaW50ZXJuYWwoVkFMVUUgZW5jb2RpbmcpCit7CisgICAgaWYgKGRl
ZmF1bHRfaW50ZXJuYWxfaW5kZXggIT0gLTIpCisJLyogQWxyZWFkeSBzZXQgKi8K
KwlyZXR1cm47CisgICAgZGVmYXVsdF9pbnRlcm5hbF9pbmRleCA9IGVuY29kaW5n
ID09IFFuaWwgPworCQkJCS0xIDpyYl9lbmNfdG9faW5kZXgocmJfdG9fZW5jb2Rp
bmcoZW5jb2RpbmcpKTsKKyAgICAvKiBDb252ZXJ0IFVTLUFTQ0lJID0+IFVURi04
ICovCisgICAgaWYgKGRlZmF1bHRfaW50ZXJuYWxfaW5kZXggPT0gcmJfdXNhc2Np
aV9lbmNpbmRleCgpKQorCWRlZmF1bHRfaW50ZXJuYWxfaW5kZXggPSByYl91dGY4
X2VuY2luZGV4KCk7CisgICAgZGVmYXVsdF9pbnRlcm5hbCA9IDA7Cit9CisKKy8q
CisgKiBjYWxsLXNlcToKICAqICAgRW5jb2RpbmcubG9jYWxlX2NoYXJtYXAgPT4g
c3RyaW5nCiAgKgogICogUmV0dXJucyB0aGUgbG9jYWxlIGNoYXJtYXAgbmFtZS4K
QEAgLTEyMTIsNiArMTI2MSw3IEBACiAgICAgcmJfZGVmaW5lX3NpbmdsZXRvbl9t
ZXRob2QocmJfY0VuY29kaW5nLCAiX2xvYWQiLCBlbmNfbG9hZCwgMSk7CiAKICAg
ICByYl9kZWZpbmVfc2luZ2xldG9uX21ldGhvZChyYl9jRW5jb2RpbmcsICJkZWZh
dWx0X2V4dGVybmFsIiwgZ2V0X2RlZmF1bHRfZXh0ZXJuYWwsIDApOworICAgIHJi
X2RlZmluZV9zaW5nbGV0b25fbWV0aG9kKHJiX2NFbmNvZGluZywgImRlZmF1bHRf
aW50ZXJuYWwiLCBnZXRfZGVmYXVsdF9pbnRlcm5hbCwgMCk7CiAgICAgcmJfZGVm
aW5lX3NpbmdsZXRvbl9tZXRob2QocmJfY0VuY29kaW5nLCAibG9jYWxlX2NoYXJt
YXAiLCByYl9sb2NhbGVfY2hhcm1hcCwgMCk7CiAKICAgICBsaXN0ID0gcmJfYXJ5
X25ldzIoZW5jX3RhYmxlLmNvdW50KTsKSW5kZXg6IGluY2x1ZGUvcnVieS9lbmNv
ZGluZy5oCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGluY2x1ZGUvcnVieS9lbmNv
ZGluZy5oCShyZXZpc2lvbiAxOTU2MSkKKysrIGluY2x1ZGUvcnVieS9lbmNvZGlu
Zy5oCSh3b3JraW5nIGNvcHkpCkBAIC0xNjgsMTEgKzE2OCwxNCBAQAogcmJfZW5j
b2RpbmcgKnJiX2xvY2FsZV9lbmNvZGluZyh2b2lkKTsKIHJiX2VuY29kaW5nICpy
Yl9maWxlc3lzdGVtX2VuY29kaW5nKHZvaWQpOwogcmJfZW5jb2RpbmcgKnJiX2Rl
ZmF1bHRfZXh0ZXJuYWxfZW5jb2Rpbmcodm9pZCk7CityYl9lbmNvZGluZyAqcmJf
ZGVmYXVsdF9pbnRlcm5hbF9lbmNvZGluZyh2b2lkKTsKIGludCByYl9hc2NpaThi
aXRfZW5jaW5kZXgodm9pZCk7CiBpbnQgcmJfdXRmOF9lbmNpbmRleCh2b2lkKTsK
IGludCByYl91c2FzY2lpX2VuY2luZGV4KHZvaWQpOwogVkFMVUUgcmJfZW5jX2Rl
ZmF1bHRfZXh0ZXJuYWwodm9pZCk7CitWQUxVRSByYl9lbmNfZGVmYXVsdF9pbnRl
cm5hbCh2b2lkKTsKIHZvaWQgcmJfZW5jX3NldF9kZWZhdWx0X2V4dGVybmFsKFZB
TFVFIGVuY29kaW5nKTsKK3ZvaWQgcmJfZW5jX3NldF9kZWZhdWx0X2ludGVybmFs
KFZBTFVFIGVuY29kaW5nKTsKIFZBTFVFIHJiX2xvY2FsZV9jaGFybWFwKFZBTFVF
IGtsYXNzKTsKIGxvbmcgcmJfbWVtc2VhcmNoKGNvbnN0IHZvaWQqLGxvbmcsY29u
c3Qgdm9pZCosbG9uZyxyYl9lbmNvZGluZyopOwogCkluZGV4OiBpby5jCj09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT0KLS0tIGlvLmMJKHJldmlzaW9uIDE5NTYxKQorKysgaW8u
Ywkod29ya2luZyBjb3B5KQpAQCAtMjE4MywxMCArMjE4Myw4IEBACiAJfQogCW5l
d2xpbmUgPSAodW5zaWduZWQgY2hhcilyc3B0cltyc2xlbiAtIDFdOwogCi0gICAg
ICAgIGlmIChmcHRyLT5lbmNzLmVuYzIpCi0gICAgICAgICAgICBlbmMgPSBmcHRy
LT5lbmNzLmVuYzsKLSAgICAgICAgZWxzZQotICAgICAgICAgICAgZW5jID0gaW9f
aW5wdXRfZW5jb2RpbmcoZnB0cik7CisJLyogTVMgLSBPcHRpbWlzYXRpb24gKi8K
KyAgICAgICAgZW5jID0gaW9fcmVhZF9lbmNvZGluZyhmcHRyKTsKIAl3aGlsZSAo
KGMgPSBhcHBlbmRsaW5lKGZwdHIsIG5ld2xpbmUsICZzdHIsICZsaW1pdCkpICE9
IEVPRikgewogICAgICAgICAgICAgY29uc3QgY2hhciAqcywgKnAsICpwcCwgKmU7
CiAKQEAgLTM3NDYsNTIgKzM3NDQsODcgQEAKICAgICByZXR1cm4gTlVMTDsJCS8q
IG5vdCByZWFjaGVkICovCiB9CiAKKy8qCisgKiBDb252ZXJ0IGV4dGVybmFsL2lu
dGVybmFsIGVuY29kaW5ncyB0byBlbmMvZW5jMgorICogTlVMTCA9PiB1c2UgZGVm
YXVsdCBlbmNvZGluZworICogUW5pbCA9PiBubyBlbmNvZGluZyBzcGVjaWZpZWQg
KGludGVybmFsIG9ubHkpCisgKi8KIHN0YXRpYyB2b2lkCityYl9pb19leHRfaW50
X3RvX2VuY3MocmJfZW5jb2RpbmcgKmV4dCwgcmJfZW5jb2RpbmcgKmludGVybiwg
cmJfZW5jb2RpbmcgKiplbmMsIHJiX2VuY29kaW5nICoqZW5jMikKK3sKKyAgICBp
bnQgZGVmYXVsdF9leHQgPSAwOworCisgICAgaWYgKGV4dCA9PSBOVUxMKSB7CisJ
ZXh0ID0gcmJfZGVmYXVsdF9leHRlcm5hbF9lbmNvZGluZygpOworCWRlZmF1bHRf
ZXh0ID0gMTsKKyAgICB9CisgICAgaWYgKGludGVybiA9PSBOVUxMICYmIGV4dCAh
PSByYl9hc2NpaThiaXRfZW5jb2RpbmcoKSkKKwkvKiBJZiBleHRlcm5hbCBpcyBB
U0NJSS04QklULCBubyBkZWZhdWx0IHRyYW5zY29kaW5nICovCisJaW50ZXJuID0g
cmJfZGVmYXVsdF9pbnRlcm5hbF9lbmNvZGluZygpOworICAgIGlmIChpbnRlcm4g
PT0gTlVMTCB8fCBpbnRlcm4gPT0gKHJiX2VuY29kaW5nICopUW5pbCB8fCBpbnRl
cm4gPT0gZXh0KSB7CisJLyogTm8gaW50ZXJuYWwgZW5jb2RpbmcgPT4gdXNlIGV4
dGVybmFsICsgbm8gdHJhbnNjb2RpbmcgKi8KKwkqZW5jID0gZGVmYXVsdF9leHQg
PyBOVUxMIDogZXh0OworCSplbmMyID0gTlVMTDsKKyAgICB9CisgICAgZWxzZSB7
CisJKmVuYyA9IGludGVybjsKKwkqZW5jMiA9IGV4dDsKKyAgICB9Cit9CisKK3N0
YXRpYyB2b2lkCiBwYXJzZV9tb2RlX2VuYyhjb25zdCBjaGFyICplc3RyLCByYl9l
bmNvZGluZyAqKmVuY19wLCByYl9lbmNvZGluZyAqKmVuYzJfcCkKIHsKLSAgICBj
b25zdCBjaGFyICpwMCwgKnAxOwotICAgIGNoYXIgKmVuYzJuYW1lOworICAgIGNv
bnN0IGNoYXIgKnA7CisgICAgY2hhciBlbmNuYW1lW0VOQ09ESU5HX01BWE5BTUVM
RU4rMV07CiAgICAgaW50IGlkeCwgaWR4MjsKKyAgICByYl9lbmNvZGluZyAqZXh0
X2VuYywgKmludF9lbmM7CiAKLSAgICAvKiBwYXJzZSBlc3RyIGFzICJlbmMiIG9y
ICJlbmMyOmVuYyIgKi8KKyAgICAvKiBwYXJzZSBlc3RyIGFzICJlbmMiIG9yICJl
bmMyOmVuYyIgb3IgImVuYzotIiAqLwogCi0gICAgKmVuY19wID0gMDsKLSAgICAq
ZW5jMl9wID0gMDsKKyAgICBwID0gc3RycmNocihlc3RyLCAnOicpOworICAgIGlm
IChwKSB7CisJaW50IGxlbiA9IChwKyspIC0gZXN0cjsKKwlpZiAobGVuID09IDAg
fHwgbGVuID4gRU5DT0RJTkdfTUFYTkFNRUxFTikKKwkgICAgaWR4ID0gLTE7CisJ
ZWxzZSB7CisJICAgIG1lbWNweShlbmNuYW1lLCBlc3RyLCBsZW4pOworCSAgICBl
bmNuYW1lW2xlbl0gPSAnXDAnOworCSAgICBlc3RyID0gZW5jbmFtZTsKKwkgICAg
aWR4ID0gcmJfZW5jX2ZpbmRfaW5kZXgoZW5jbmFtZSk7CisJfQorICAgIH0KKyAg
ICBlbHNlCisJaWR4ID0gcmJfZW5jX2ZpbmRfaW5kZXgoZXN0cik7CiAKLSAgICBw
MCA9IHN0cnJjaHIoZXN0ciwgJzonKTsKLSAgICBpZiAoIXAwKSBwMSA9IGVzdHI7
Ci0gICAgZWxzZSAgICAgcDEgPSBwMCArIDE7Ci0gICAgaWR4ID0gcmJfZW5jX2Zp
bmRfaW5kZXgocDEpOwotICAgIGlmIChpZHggPj0gMCkgewotCSplbmNfcCA9IHJi
X2VuY19mcm9tX2luZGV4KGlkeCk7Ci0gICAgfQorICAgIGlmIChpZHggPj0gMCkK
KwlleHRfZW5jID0gcmJfZW5jX2Zyb21faW5kZXgoaWR4KTsKICAgICBlbHNlIHsK
LQlyYl93YXJuKCJVbnN1cHBvcnRlZCBlbmNvZGluZyAlcyBpZ25vcmVkIiwgcDEp
OworCWlmIChpZHggIT0gLTIpCisJICAgIHJiX3dhcm4oIlVuc3VwcG9ydGVkIGVu
Y29kaW5nICVzIGlnbm9yZWQiLCBlc3RyKTsKKwlleHRfZW5jID0gTlVMTDsKICAg
ICB9CiAKLSAgICBpZiAoKmVuY19wICYmIHAwKSB7Ci0JaW50IG4gPSBwMCAtIGVz
dHI7Ci0JaWYgKG4gPiBFTkNPRElOR19NQVhOQU1FTEVOKSB7Ci0JICAgIGlkeDIg
PSAtMTsKKyAgICBpbnRfZW5jID0gTlVMTDsKKyAgICBpZiAocCkgeworCWlmICgq
cCA9PSAnLScgJiYgKihwKzEpID09ICdcMCcpIHsKKwkgICAgLyogU3BlY2lhbCBj
YXNlIC0gIi0iID0+IG5vIHRyYW5zY29kaW5nICovCisJICAgIGludF9lbmMgPSAo
cmJfZW5jb2RpbmcgKilRbmlsOwogCX0KIAllbHNlIHsKLQkgICAgZW5jMm5hbWUg
PSBBTExPQ0FfTihjaGFyLCBuKzEpOwotCSAgICBtZW1jcHkoZW5jMm5hbWUsIGVz
dHIsIG4pOwotCSAgICBlbmMybmFtZVtuXSA9ICdcMCc7Ci0JICAgIGVzdHIgPSBl
bmMybmFtZTsKLQkgICAgaWR4MiA9IHJiX2VuY19maW5kX2luZGV4KGVuYzJuYW1l
KTsKKwkgICAgaWR4MiA9IHJiX2VuY19maW5kX2luZGV4KHApOworCSAgICBpZiAo
aWR4MiA8IDApCisJCXJiX3dhcm4oIlVuc3VwcG9ydGVkIGVuY29kaW5nICVzIGln
bm9yZWQiLCBwKTsKKwkgICAgZWxzZSBpZiAoaWR4MiA9PSBpZHgpIHsKKwkJcmJf
d2FybigiSWdub3JpbmcgaW50ZXJuYWwgZW5jb2RpbmcgJXM6IGl0IGlzIGlkZW50
aWNhbCB0byBleHRlcm5hbCBlbmNvZGluZyAlcyIsIHAsIGVzdHIpOworCQlpbnRf
ZW5jID0gKHJiX2VuY29kaW5nICopUW5pbDsKKwkgICAgfQorCSAgICBlbHNlCisJ
CWludF9lbmMgPSByYl9lbmNfZnJvbV9pbmRleChpZHgyKTsKIAl9Ci0JaWYgKGlk
eDIgPCAwKSB7Ci0JICAgIHJiX3dhcm4oIlVuc3VwcG9ydGVkIGVuY29kaW5nICUu
KnMgaWdub3JlZCIsIG4sIGVzdHIpOwotCX0KLQllbHNlIGlmIChpZHgyID09IGlk
eCkgewotCSAgICByYl93YXJuKCJJZ25vcmluZyBpbnRlcm5hbCBlbmNvZGluZyAl
LipzOiBpdCBpcyBpZGVudGljYWwgdG8gZXh0ZXJuYWwgZW5jb2RpbmcgJXMiLAot
CQkgICAgbiwgZXN0ciwgcDEpOwotCX0KLQllbHNlIHsKLQkgICAgKmVuYzJfcCA9
IHJiX2VuY19mcm9tX2luZGV4KGlkeDIpOwotCX0KICAgICB9CisKKyAgICByYl9p
b19leHRfaW50X3RvX2VuY3MoZXh0X2VuYywgaW50X2VuYywgZW5jX3AsIGVuYzJf
cCk7CiB9CiAKIHN0YXRpYyB2b2lkCkBAIC0zODI3LDI4ICszODYwLDMyIEBACiAg
ICAgfQogICAgIGlmICghTklMX1AoZXh0ZW5jKSkgewogCXJiX2VuY29kaW5nICpl
eHRlbmNvZGluZyA9IHJiX3RvX2VuY29kaW5nKGV4dGVuYyk7CisJcmJfZW5jb2Rp
bmcgKmludGVuY29kaW5nID0gTlVMTDsKICAgICAgICAgZXh0cmFjdGVkID0gMTsK
LSAgICAgICAgKmVuY19wID0gMDsKLSAgICAgICAgKmVuYzJfcCA9IDA7CiAJaWYg
KCFOSUxfUChlbmNvZGluZykpIHsKIAkgICAgcmJfd2FybigiSWdub3JpbmcgZW5j
b2RpbmcgcGFyYW1ldGVyICclcyc6IGV4dGVybmFsX2VuY29kaW5nIGlzIHVzZWQi
LAogCQkgICAgUlNUUklOR19QVFIoZW5jb2RpbmcpKTsKIAl9CiAJaWYgKCFOSUxf
UChpbnRlbmMpKSB7Ci0JICAgIHJiX2VuY29kaW5nICppbnRlbmNvZGluZyA9IHJi
X3RvX2VuY29kaW5nKGludGVuYyk7CisJICAgIGlmICghTklMX1AoZW5jb2Rpbmcg
PSByYl9jaGVja19zdHJpbmdfdHlwZShpbnRlbmMpKSkgeworCQljaGFyICpwID0g
U3RyaW5nVmFsdWVDU3RyKGVuY29kaW5nKTsKKwkJaWYgKCpwID09ICctJyAmJiAq
KHArMSkgPT0gJ1wwJykgeworCQkgICAgLyogU3BlY2lhbCBjYXNlIC0gIi0iID0+
IG5vIHRyYW5zY29kaW5nICovCisJCSAgICBpbnRlbmNvZGluZyA9IChyYl9lbmNv
ZGluZyAqKVFuaWw7CisJCX0KKwkJZWxzZQorCQkgICAgaW50ZW5jb2RpbmcgPSBy
Yl90b19lbmNvZGluZyhpbnRlbmMpOworCSAgICB9CisJICAgIGVsc2UKKwkJaW50
ZW5jb2RpbmcgPSByYl90b19lbmNvZGluZyhpbnRlbmMpOwogCSAgICBpZiAoZXh0
ZW5jb2RpbmcgPT0gaW50ZW5jb2RpbmcpIHsKIAkJcmJfd2FybigiSWdub3Jpbmcg
aW50ZXJuYWwgZW5jb2RpbmcgJyVzJzogaXQgaXMgaWRlbnRpY2FsIHRvIGV4dGVy
bmFsIGVuY29kaW5nICclcyciLAogCQkJUlNUUklOR19QVFIocmJfaW5zcGVjdChp
bnRlbmMpKSwKIAkJCVJTVFJJTkdfUFRSKHJiX2luc3BlY3QoZXh0ZW5jKSkpOwor
CQlpbnRlbmNvZGluZyA9IChyYl9lbmNvZGluZyAqKVFuaWw7CiAJICAgIH0KLQkg
ICAgZWxzZSB7Ci0JCSplbmNfcCA9IGludGVuY29kaW5nOwotICAgICAgICAgICAg
ICAgICplbmMyX3AgPSBleHRlbmNvZGluZzsKLQkgICAgfQogCX0KLSAgICAgICAg
ZWxzZSB7Ci0gICAgICAgICAgICAqZW5jX3AgPSBleHRlbmNvZGluZzsKLSAgICAg
ICAgfQorCXJiX2lvX2V4dF9pbnRfdG9fZW5jcyhleHRlbmNvZGluZywgaW50ZW5j
b2RpbmcsIGVuY19wLCBlbmMyX3ApOwogICAgIH0KICAgICBlbHNlIHsKIAlpZiAo
IU5JTF9QKGludGVuYykpIHsKQEAgLTM4ODgsOCArMzkyNSw4IEBACiAKICAgICB2
bW9kZSA9ICp2bW9kZV9wOwogCi0gICAgZW5jID0gTlVMTDsKLSAgICBlbmMyID0g
TlVMTDsKKyAgICAvKiBTZXQgdG8gZGVmYXVsdHMgKi8KKyAgICByYl9pb19leHRf
aW50X3RvX2VuY3MoTlVMTCwgTlVMTCwgJmVuYywgJmVuYzIpOwogCiAgICAgaWYg
KE5JTF9QKHZtb2RlKSkgewogICAgICAgICBmbW9kZSA9IEZNT0RFX1JFQURBQkxF
OwpAQCAtNDA3Niw4ICs0MTEzLDggQEAKICAgICByYl9pb190ICpmcHRyOwogICAg
IGNvbnZjb25maWdfdCBjYzsKICAgICBpZiAoIWNvbnZjb25maWcpIHsKLSAgICAg
ICAgY2MuZW5jID0gTlVMTDsKLSAgICAgICAgY2MuZW5jMiA9IE5VTEw7CisJLyog
U2V0IHRvIGRlZmF1bHQgZW5jb2RpbmdzICovCisJcmJfaW9fZXh0X2ludF90b19l
bmNzKE5VTEwsIE5VTEwsICZjYy5lbmMsICZjYy5lbmMyKTsKICAgICAgICAgY2Mu
ZWNmbGFncyA9IDA7CiAgICAgICAgIGNjLmVjb3B0cyA9IFFuaWw7CiAgICAgICAg
IGNvbnZjb25maWcgPSAmY2M7CkBAIC00MTA1LDggKzQxNDIsOCBAQAogICAgICAg
ICBwYXJzZV9tb2RlX2VuYyhwKzEsICZjb252Y29uZmlnLmVuYywgJmNvbnZjb25m
aWcuZW5jMik7CiAgICAgfQogICAgIGVsc2UgewotICAgICAgICBjb252Y29uZmln
LmVuYyA9IE5VTEw7Ci0gICAgICAgIGNvbnZjb25maWcuZW5jMiA9IE5VTEw7CisJ
LyogU2V0IHRvIGRlZmF1bHQgZW5jb2RpbmdzICovCisJcmJfaW9fZXh0X2ludF90
b19lbmNzKE5VTEwsIE5VTEwsICZjb252Y29uZmlnLmVuYywgJmNvbnZjb25maWcu
ZW5jMik7CiAgICAgICAgIGNvbnZjb25maWcuZWNmbGFncyA9IDA7CiAgICAgICAg
IGNvbnZjb25maWcuZWNvcHRzID0gUW5pbDsKICAgICB9CkBAIC02Njc3LDI5ICs2
NzE0LDQwIEBACiB7CiAgICAgcmJfZW5jb2RpbmcgKmVuYywgKmVuYzI7CiAgICAg
aW50IGVjZmxhZ3M7Ci0gICAgVkFMVUUgZWNvcHRzOworICAgIFZBTFVFIGVjb3B0
cywgdG1wOwogCiAgICAgaWYgKCFOSUxfUCh2MikpIHsKIAllbmMyID0gcmJfdG9f
ZW5jb2RpbmcodjEpOwotCWVuYyA9IHJiX3RvX2VuY29kaW5nKHYyKTsKKwl0bXAg
PSByYl9jaGVja19zdHJpbmdfdHlwZSh2Mik7CisJaWYgKCFOSUxfUCh0bXApKSB7
CisJICAgIGNoYXIgKnAgPSBTdHJpbmdWYWx1ZUNTdHIodG1wKTsKKwkgICAgaWYg
KCpwID09ICctJyAmJiAqKHArMSkgPT0gJ1wwJykgeworCQkvKiBTcGVjaWFsIGNh
c2UgLSAiLSIgPT4gbm8gdHJhbnNjb2RpbmcgKi8KKwkJZW5jID0gZW5jMjsKKwkJ
ZW5jMiA9IE5VTEw7CisJICAgIH0KKwkgICAgZWxzZQorCQllbmMgPSByYl90b19l
bmNvZGluZyh2Mik7CisJfQorCWVsc2UKKwkgICAgZW5jID0gcmJfdG9fZW5jb2Rp
bmcodjIpOwogICAgICAgICBlY2ZsYWdzID0gcmJfZWNvbnZfcHJlcGFyZV9vcHRz
KG9wdCwgJmVjb3B0cyk7CiAgICAgfQogICAgIGVsc2UgewogCWlmIChOSUxfUCh2
MSkpIHsKLQkgICAgZW5jID0gTlVMTDsKLQkgICAgZW5jMiA9IE5VTEw7CisJICAg
IC8qIFNldCB0byBkZWZhdWx0IGVuY29kaW5ncyAqLworCSAgICByYl9pb19leHRf
aW50X3RvX2VuY3MoTlVMTCwgTlVMTCwgJmVuYywgJmVuYzIpOwogICAgICAgICAg
ICAgZWNmbGFncyA9IDA7CiAgICAgICAgICAgICBlY29wdHMgPSBRbmlsOwogCX0K
IAllbHNlIHsKLQkgICAgVkFMVUUgdG1wID0gcmJfY2hlY2tfc3RyaW5nX3R5cGUo
djEpOworCSAgICB0bXAgPSByYl9jaGVja19zdHJpbmdfdHlwZSh2MSk7CiAJICAg
IGlmICghTklMX1AodG1wKSkgewogICAgICAgICAgICAgICAgIHBhcnNlX21vZGVf
ZW5jKFN0cmluZ1ZhbHVlQ1N0cih0bXApLCAmZW5jLCAmZW5jMik7CiAgICAgICAg
ICAgICAgICAgZWNmbGFncyA9IHJiX2Vjb252X3ByZXBhcmVfb3B0cyhvcHQsICZl
Y29wdHMpOwogCSAgICB9CiAJICAgIGVsc2UgewotCQllbmMgPSByYl90b19lbmNv
ZGluZyh2MSk7Ci0JCWVuYzIgPSBOVUxMOworCQlyYl9pb19leHRfaW50X3RvX2Vu
Y3MocmJfdG9fZW5jb2RpbmcodjEpLCBOVUxMLCAmZW5jLCAmZW5jMik7CiAgICAg
ICAgICAgICAgICAgZWNmbGFncyA9IDA7CiAgICAgICAgICAgICAgICAgZWNvcHRz
ID0gUW5pbDsKIAkgICAgfQpJbmRleDogcGFyc2UueQo9PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09Ci0tLSBwYXJzZS55CShyZXZpc2lvbiAxOTU2MSkKKysrIHBhcnNlLnkJKHdv
cmtpbmcgY29weSkKQEAgLTYwNjgsNiArNjA2OCw4IEBACiAgICAgaWYgKHBhcnNl
ci0+bGluZV9jb3VudCAhPSAocGFyc2VyLT5oYXNfc2hlYmFuZyA/IDIgOiAxKSkK
IAlyZXR1cm47CiAgICAgcGFyc2VyX3NldF9lbmNvZGUocGFyc2VyLCB2YWwpOwor
ICAgIGlmIChzdHJjbXAobmFtZSwgImludGVybmFsX2VuY29kaW5nIikgPT0gMCkK
KwlyYl9lbmNfc2V0X2RlZmF1bHRfaW50ZXJuYWwocmJfZW5jX2Zyb21fZW5jb2Rp
bmcocGFyc2VyLT5lbmMpKTsKIH0KIAogc3RydWN0IG1hZ2ljX2NvbW1lbnQgewpA
QCAtNjA3OSw2ICs2MDgxLDcgQEAKIHN0YXRpYyBjb25zdCBzdHJ1Y3QgbWFnaWNf
Y29tbWVudCBtYWdpY19jb21tZW50c1tdID0gewogICAgIHsiY29kaW5nIiwgbWFn
aWNfY29tbWVudF9lbmNvZGluZywgcGFyc2VyX2VuY29kZV9sZW5ndGh9LAogICAg
IHsiZW5jb2RpbmciLCBtYWdpY19jb21tZW50X2VuY29kaW5nLCBwYXJzZXJfZW5j
b2RlX2xlbmd0aH0sCisgICAgeyJpbnRlcm5hbF9lbmNvZGluZyIsIG1hZ2ljX2Nv
bW1lbnRfZW5jb2RpbmcsIHBhcnNlcl9lbmNvZGVfbGVuZ3RofSwKIH07CiAjZW5k
aWYKIApAQCAtNjIwNyw2ICs2MjEwLDggQEAKIHsKICAgICBpbnQgc2VwID0gMDsK
ICAgICBjb25zdCBjaGFyICpiZWcgPSBzdHI7CisgICAgY29uc3QgY2hhciAqbmFt
ZTsKKyAgICBpbnQgbmFtZV9sZW47CiAgICAgVkFMVUUgczsKIAogICAgIGZvciAo
OzspIHsKQEAgLTYyMjksNiArNjIzNCwxMSBAQAogCX0KIAlpZiAoU1RSTkNBU0VD
TVAoc3RyLTYsICJjb2RpbmciLCA2KSA9PSAwKSBicmVhazsKICAgICB9CisgICAg
LyogU2VhcmNoIGZvciB0aGUgc3RhcnQgb2YgdGhlIGtleXdvcmQgKi8KKyAgICBm
b3IgKG5hbWUgPSBzdHItNjsgbmFtZSA+PSBiZWc7IG5hbWUtLSkKKwlpZiAoIUlT
QUxOVU0oKm5hbWUpICYmICpuYW1lICE9ICdfJykKKwkgICAgYnJlYWs7CisgICAg
bmFtZV9sZW4gPSBzdHIgLSArK25hbWU7CiAgICAgZm9yICg7OykgewogCWRvIHsK
IAkgICAgaWYgKCsrc3RyID49IHNlbmQpIHJldHVybjsKQEAgLTYyNDMsNiArNjI1
Myw4IEBACiAgICAgcyA9IHJiX3N0cl9uZXcoYmVnLCBwYXJzZXJfZW5jb2RlX2xl
bmd0aChwYXJzZXIsIGJlZywgc3RyIC0gYmVnKSk7CiAgICAgcGFyc2VyX3NldF9l
bmNvZGUocGFyc2VyLCBSU1RSSU5HX1BUUihzKSk7CiAgICAgcmJfc3RyX3Jlc2l6
ZShzLCAwKTsKKyAgICBpZiAobmFtZV9sZW4gPT0gMTcgJiYgU1RSTkNBU0VDTVAo
bmFtZSwgImludGVybmFsX2VuY29kaW5nIiwgMTcpID09IDApCisJcmJfZW5jX3Nl
dF9kZWZhdWx0X2ludGVybmFsKHJiX2VuY19mcm9tX2VuY29kaW5nKHBhcnNlci0+
ZW5jKSk7CiB9CiAKIHN0YXRpYyB2b2lkCkluZGV4OiBydWJ5LmMKPT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PQotLS0gcnVieS5jCShyZXZpc2lvbiAxOTU2MSkKKysrIHJ1Ynku
Ywkod29ya2luZyBjb3B5KQpAQCAtOTQsNyArOTQsNyBAQAogCSAgICBWQUxVRSBu
YW1lOwogCSAgICBpbnQgaW5kZXg7CiAJfSBlbmM7Ci0gICAgfSBzcmMsIGV4dDsK
KyAgICB9IHNyYywgZXh0LCBpbnRlcm47CiAgICAgVkFMVUUgcmVxX2xpc3Q7CiB9
OwogCkBAIC04NjksNiArODY5LDcgQEAKIAkJcnVieV9lYWNoX3dvcmRzKHMsIGRp
c2FibGVfb3B0aW9uLCAmb3B0LT5kaXNhYmxlKTsKIAkgICAgfQogCSAgICBlbHNl
IGlmIChzdHJuY21wKCJlbmNvZGluZyIsIHMsIG4gPSA4KSA9PSAwICYmICghc1tu
XSB8fCBzW25dID09ICc9JykpIHsKKwkJY2hhciAqcDsKIAkJcyArPSBuOwogCQlp
ZiAoISpzKyspIHsKIAkJICBuZXh0X2VuY29kaW5nOgpAQCAtODc3LDcgKzg3OCwx
NSBAQAogCQkgICAgfQogCQl9CiAJICAgICAgZW5jb2Rpbmc6Ci0JCW9wdC0+ZXh0
LmVuYy5uYW1lID0gcmJfc3RyX25ldzIocyk7CisJCXAgPSBzdHJjaHIocywgJzon
KTsKKwkJaWYgKHApIHsKKwkJICAgIGlmIChwID4gcykKKwkJCW9wdC0+ZXh0LmVu
Yy5uYW1lID0gcmJfc3RyX25ldyhzLCBwLXMpOworCQkgICAgaWYgKCorK3ApCisJ
CQlvcHQtPmludGVybi5lbmMubmFtZSA9IHJiX3N0cl9uZXcyKHApOworCQl9CisJ
CWVsc2UgICAgCisJCSAgICBvcHQtPmV4dC5lbmMubmFtZSA9IHJiX3N0cl9uZXcy
KHMpOwogCSAgICB9CiAJICAgIGVsc2UgaWYgKHN0cmNtcCgidmVyc2lvbiIsIHMp
ID09IDApCiAJCW9wdC0+dmVyc2lvbiA9IDE7CkBAIC05ODAsNiArOTg5LDcgQEAK
IAlyYl9zYWZlX2xldmVsKCkgPT0gMCAmJiAocyA9IGdldGVudigiUlVCWU9QVCIp
KSkgewogCVZBTFVFIHNyY19lbmNfbmFtZSA9IG9wdC0+c3JjLmVuYy5uYW1lOwog
CVZBTFVFIGV4dF9lbmNfbmFtZSA9IG9wdC0+ZXh0LmVuYy5uYW1lOworCVZBTFVF
IGludF9lbmNfbmFtZSA9IG9wdC0+aW50ZXJuLmVuYy5uYW1lOwogCiAJd2hpbGUg
KElTU1BBQ0UoKnMpKQogCSAgICBzKys7CkBAIC0xMDE5LDYgKzEwMjksOCBAQAog
CSAgICBvcHQtPnNyYy5lbmMubmFtZSA9IHNyY19lbmNfbmFtZTsKIAlpZiAoZXh0
X2VuY19uYW1lKQogCSAgICBvcHQtPmV4dC5lbmMubmFtZSA9IGV4dF9lbmNfbmFt
ZTsKKwlpZiAoaW50X2VuY19uYW1lKQorCSAgICBvcHQtPmludGVybi5lbmMubmFt
ZSA9IGludF9lbmNfbmFtZTsKICAgICB9CiAKICAgICBpZiAob3B0LT52ZXJzaW9u
KSB7CkBAIC0xMDg3LDYgKzEwOTksOSBAQAogICAgIGlmIChvcHQtPmV4dC5lbmMu
bmFtZSAhPSAwKSB7CiAJb3B0LT5leHQuZW5jLmluZGV4ID0gb3B0X2VuY19pbmRl
eChvcHQtPmV4dC5lbmMubmFtZSk7CiAgICAgfQorICAgIGlmIChvcHQtPmludGVy
bi5lbmMubmFtZSAhPSAwKSB7CisJb3B0LT5pbnRlcm4uZW5jLmluZGV4ID0gb3B0
X2VuY19pbmRleChvcHQtPmludGVybi5lbmMubmFtZSk7CisgICAgfQogICAgIGlm
IChvcHQtPnNyYy5lbmMubmFtZSAhPSAwKSB7CiAJb3B0LT5zcmMuZW5jLmluZGV4
ID0gb3B0X2VuY19pbmRleChvcHQtPnNyYy5lbmMubmFtZSk7CiAJc3JjX2VuY29k
aW5nX2luZGV4ID0gb3B0LT5zcmMuZW5jLmluZGV4OwpAQCAtMTA5OCw2ICsxMTEz
LDExIEBACiAJZW5jID0gbGVuYzsKICAgICB9CiAgICAgcmJfZW5jX3NldF9kZWZh
dWx0X2V4dGVybmFsKHJiX2VuY19mcm9tX2VuY29kaW5nKGVuYykpOworICAgIGlm
IChvcHQtPmludGVybi5lbmMuaW5kZXggPj0gMCkgeworCWVuYyA9IHJiX2VuY19m
cm9tX2luZGV4KG9wdC0+aW50ZXJuLmVuYy5pbmRleCk7CisJcmJfZW5jX3NldF9k
ZWZhdWx0X2ludGVybmFsKHJiX2VuY19mcm9tX2VuY29kaW5nKGVuYykpOworCW9w
dC0+aW50ZXJuLmVuYy5pbmRleCA9IC0xOworICAgIH0KIAogICAgIHJiX3NldF9z
YWZlX2xldmVsX2ZvcmNlKHNhZmUpOwogICAgIGlmIChvcHQtPmVfc2NyaXB0KSB7
CkBAIC0xMTE5LDYgKzExMzksMTUgQEAKIAl0cmVlID0gbG9hZF9maWxlKHBhcnNl
ciwgb3B0LT5zY3JpcHQsIDEsIG9wdCk7CiAgICAgfQogCisgICAgaWYgKG9wdC0+
aW50ZXJuLmVuYy5pbmRleCA+PSAwKSB7CisJLyogU2V0IGluIHRoZSBzaGViYW5n
IGxpbmUgKi8KKwllbmMgPSByYl9lbmNfZnJvbV9pbmRleChvcHQtPmludGVybi5l
bmMuaW5kZXgpOworCXJiX2VuY19zZXRfZGVmYXVsdF9pbnRlcm5hbChyYl9lbmNf
ZnJvbV9lbmNvZGluZyhlbmMpKTsKKyAgICB9CisgICAgZWxzZQorCS8qIEZyZWV6
ZSBkZWZhdWx0X2ludGVybmFsICovCisJcmJfZW5jX3NldF9kZWZhdWx0X2ludGVy
bmFsKFFuaWwpOworCiAgICAgaWYgKCF0cmVlKSByZXR1cm4gUWZhbHNlOwogCiAg
ICAgcHJvY2Vzc19zZmxhZyhvcHQpOwpAQCAtMTE4OSw2ICsxMjE4LDcgQEAKIAlj
aGFyICpwOwogCWludCBub19zcmNfZW5jID0gIW9wdC0+c3JjLmVuYy5uYW1lOwog
CWludCBub19leHRfZW5jID0gIW9wdC0+ZXh0LmVuYy5uYW1lOworCWludCBub19p
bnRfZW5jID0gIW9wdC0+aW50ZXJuLmVuYy5uYW1lOwogCiAJZW5jID0gcmJfdXNh
c2NpaV9lbmNvZGluZygpOwogCXJiX2Z1bmNhbGwoZiwgcmJfaW50ZXJuKCJzZXRf
ZW5jb2RpbmciKSwgMSwgcmJfZW5jX2Zyb21fZW5jb2RpbmcoZW5jKSk7CkBAIC0x
Mjc1LDYgKzEzMDUsOSBAQAogCSAgICBpZiAobm9fZXh0X2VuYyAmJiBvcHQtPmV4
dC5lbmMubmFtZSkgewogCQlvcHQtPmV4dC5lbmMuaW5kZXggPSBvcHRfZW5jX2lu
ZGV4KG9wdC0+ZXh0LmVuYy5uYW1lKTsKIAkgICAgfQorCSAgICBpZiAobm9faW50
X2VuYyAmJiBvcHQtPmludGVybi5lbmMubmFtZSkgeworCQlvcHQtPmludGVybi5l
bmMuaW5kZXggPSBvcHRfZW5jX2luZGV4KG9wdC0+aW50ZXJuLmVuYy5uYW1lKTsK
KwkgICAgfQogCX0KIAllbHNlIGlmICghTklMX1AoYykpIHsKIAkgICAgcmJfaW9f
dW5nZXRieXRlKGYsIGMpOwpAQCAtMTUzOCw2ICsxNTcxLDcgQEAKICAgICBhcmdz
LmFyZ3YgPSBhcmd2OwogICAgIGFyZ3Mub3B0ID0gY21kbGluZV9vcHRpb25zX2lu
aXQoJm9wdCk7CiAgICAgb3B0LmV4dC5lbmMuaW5kZXggPSAtMTsKKyAgICBvcHQu
aW50ZXJuLmVuYy5pbmRleCA9IC0xOwogICAgIHRyZWUgPSAoTk9ERSAqKXJiX3Zt
X2NhbGxfY2Z1bmMocmJfdm1fdG9wX3NlbGYoKSwKIAkJCQkgICAgcHJvY2Vzc19v
cHRpb25zLCAoVkFMVUUpJmFyZ3MsCiAJCQkJICAgIDAsIHJiX3Byb2duYW1lKTsK
------------GbwtIOsuhVxnZ0s04hHuwa--