Issue #13110 has been updated by Martin D端rst.


Benoit Daloze wrote:
> Shugo Maeda wrote:
 
> > UTF-32 is not useful because it's a dummy encoding.
> 
> What about UTF-32BE or UTF-32LE?

Yes, that's what I meant.

Shugo Maeda wrote:
> Martin D端rst wrote:
> > Shugo Maeda wrote:

> > But what I'm wondering is that using a single string for the data in an editor buffer may still be quite inefficient. Adding or deleting a character in the middle of the buffer will be slow, even if you know the exact position in bytes. Changing the representation e.g. to an array of lines will make the efficiency mostly go away. (After all, editors need only be as fast as humans can type :-).
> 
> I use a technique called buffer gap described in "The Craft of Text Editing" to improve performance.
>   https://www.finseth.com/craft/
> See Chapter 6 of the book for details.

The "buffer gap" technique is very well known, I'm familiar with it since the early 90ies. I was thinking about it, but I think it won't work with UTF-8. If you have figured out how you would make it work with UTF-8, then please tell us.

Here is why I think it won't work with UTF-8. The problem is that you can't move characters from before the gap to after or the other way round and change them when there are edits. If some characters are changed, they might change their byte length. But if you want to keep the string as valid UTF-8, you have to constantly fix the content of the gap. One could imagine using two separate String objects, one for before the gap and one for after. For before the gap, it actually might work quite well (as long as Ruby doesn't shorten the memory allocated to a string when the string contents is truncated), but for after the gap, it won't work, because every insertion or deletion at the end of the gap will make the string contents shift around.

> > More generally, what I'm afraid of is that with this, we start to more and more expose String internals. That can easily lead to problems.
> > 
> > Some people may copy a Ruby snippet using byteindex, then add 1 to that index because they think that's how to get to the next character. Others may start to use byteindex everywhere, even if it's absolutely not necessary. Others may demand byte- versions of more and more operations on strings. We have seen all of this in other contexts.
> 
> Doesn't this concern apply to `byteslice`?

Yes, it does. The less we have of such kinds of methods, the better.

Anyway, one more question: Are you really having performance problems, or are you just worried about performance? Compared to today's hardware speed, human editing is extremely slow, and for most operations, there should be on delay whatever.

Regards,   Martin.

----------------------------------------
Feature #13110: Byte-based operations for String
https://bugs.ruby-lang.org/issues/13110#change-62617

* Author: Shugo Maeda
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
How about to add byte-based operations for String?

```ruby
s = "
p s.byteindex(/ああ/, 4) #=> 18
x, y = Regexp.last_match.byteoffset(0) #=> [18, 24]
s.bytesplice(x...y, "おおお")
p s #=> "あああいいいおおおあ"
```



---Files--------------------------------
byteindex.diff (2.83 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>