"Christian" <christians / syd.microforte.com.au> wrote:
>
>Hi Ben,

Hi. :-)

>"Ben Tilly" <ben_tilly / hotmail.com> wrote:
>
> > I submit that C++ offers type-safety.  And then offers
> > quite sophisticated generic programming techniques that in
> > the real world are mostly used to get around issues that a
> > dynamic [-ally typed system] avoids by putting off the type-check later.
>
>Saying that everything is an object says nothing. Saying that you can have 
>a
>'generic container' because you can make a list that stores Object
>references is just plain stupid. You are just passing around pair<void *,
>type>.

Agreed that if everything is an object, that statement has
zero semantic content.  But it is a statement with
syntactic content.  It says that anything admits of certain
types of operations whenever you want.

>Consider Java's 'generic' container system. They are horrible. You can add
>*anything* to such containers (except built-in types!). Then, to be sure,
>you *must* test the type of each object in the container as you traverse 
>it.
>That makes ugly, slow code. Sure, it is 'dynamically type safe'. But it is
>ugly, slow code.

Why must you test?  Testing takes time and by doing it
you are assuming that you know in advance what types
will be able to behave in advance.  Should that
assumption be sprinkled through your code?  (OK, perhaps
in Java.)

>To clarify, I am arguing against the position that templates are simply a
>means to 'get around' the lack of 'dynamic typing'. OTOH templates (more
>accurately, static typing) remove the need for having to manually test type
>at runtime, so in that sense it is true.

I don't think you are arguing against my position.  My
position was that templates are often *used* that way.
Restating that, within a dynamic language you often use
the dynamicness of the language to solve problems which
would have required some sort of template solution in
another language.

This does not deny the existence of real problems which
templates solve that dynamicness does not.

[...]
> > And when you use these features of C++, what happens?  Why
> > the executable gets very bloated.  But just have the
> > facility to do it dynamically at runtime and for a little
> > overhead you get a much smaller process image...
>
>Not true. Sure, templates allow you to stamp out huge masses of code
>needlessly (the most evil thing you can do is store pointers in 
>containers).
>But no-one can stop you from stubbing your toe, and in any case you
>shouldn't blame the rock that you kicked. C++ now has RTTI, so you can do 
>it
>at runtime when you can't do it at compile time. I am growing weary of
>defending/propounding C++.

Theoretically you may be able to prevent it.  But in
practice I believe that it happens a lot.

>Slightly off-topic (hey, it hasn't stopped me before): why can we not have
>an interpreted language that is based on C++, not C? Python is C, Ruby is 
>C,
>Perl is C. What about a language that is built using C++ and C++ 
>principals,
>where we can extend and enhance in C++? Is there one?

Do you want my guess?

Good C compilers are more widely used than good C++
compilers, plus you are more likely to be successful
linking binaries built by different C compilers than
C++ compilers.  Considering that one of the key things
that interpreted languages are used for is to put
scriptable interfaces around existing libraries, these
two issues are big.

[...]
> > The only way I learn it seems is to form an opinion and
> > then be vocal about it in a forum where knowledgable
> > people are to be found.  [...]
>
>Hehe, yes that is effectively what I have done, consciously or not. To drop
>to a personal level momentarily, I find that my collegues are intelligent
>but not very interested in talking about deeper issues. So I jumped in feet
>first, and have enjoyed the refreshing water immensely.

Been there, done that.  Isn't it effective? :-)

[...]
> > >It is exactly the attitude that languages like Ruby make anything
>fundamentally 'easier' that
> > >fires me up to write posts like those I made previously.
> >
> > Define easier.
>
>What I meant was that it is a trap to think that any language is a silver
>bullet. I have previously claimed that there is not much need for anything
>other than C++ (given Managed C++). Of course, I was being facetious (the 
>/.
>term is troll).

No language is a silver bullet.  Indeed the claims you hear
from Python people that its syntax is a silver bullet is
one (of many) things that irritates me about Python.

[...]
> > I submit that easier depends on both the problem and the
> > person attempting to solve that problem.
>
>True enough.
>
> > I also submit that good programmers tend to outright get
> > their accounting wrong more often than overall designs.
>
>Absolutely not true :). Spelling is easy, composition is difficult and
>requires maturity, experience, tools and intelligence. There are many 
>'good'
>programmers that implement bad designs really well.

Data point: Among competent C programmers the most common
error is believed to be off-by-one.

Data point: Competent Ruby (Perl, etc) programmers almost
never have any off-by-one errors.  Why not?  Because the
language has a native "foreach" construct which you are
encouraged to use.  Use that and you simply cannot make
an off-by-one error.

Data point: The most common security hole is the buffer
overflow.

Data point: In languages with automatic garbage
collection it is (virtually) impossible to have a buffer
overflow.

I stand by my belief that human brains are not well
suited to doing accounting and in languages where you
can shove the work of doing accounting to the language
you get immediate and large benefits.  Specifically the
code tends to be easier to write, simpler to understand,
and contains fewer errors.  And sure, the errors may be
trivial.  But that doesn't make a buffer overflow
particularly trivial to track down...

> > Conversely when you make an accounting error it tends to
> > be harder to spot than code which did not fit in your
> > overall design.
>
>I had trouble understanding this. Are you saying that off-by-one errors are
>harder to identify than superfluous systems? The search spaces are
>unrelated.

What I meant is that if you call a method that doesn't
exist it is easy for the language to report what you did
with enough context to immediately see the problem.
With an off-by-one error it is not as obvious.

> > Languages like Ruby are well-designed to support styles
> > of programming where you do less explicit accounting and
> > more of your work at the overall design.
>
>True enough. But in my experience permissive languages like Ruby are,
>paradoxically, better suited for experts than novices, although this is not
>an obvious point. You need to understand type before you can ignore it. 
>Some
>of the most excellent programmers I know are Python hacks. They can write 
>20
>lines of code that are so deep it takes me days to understand it.

Agreed.  Too many people when they find that they can get
things done without discipline don't bother.  The quote
that I like for this from Perl is, "There is no fundamental
difference between scripting Perl and programming.  But the
people who call it programming generally do a better job."

The trade-off is between having your most competent people
being able to do more and having them be able to direct
less capable people.  My belief is that the former works
better because of the problems in scaling projects noticed
as far back as "The Mythical Man-Month".  (The correlation
between the size of programming projects and failure rates
is simply astonishing.)

A tangential but interesting discussion on a related topic
you might enjoy:

http://pub13.ezboard.com/fiwetheylinux.showMessage?topicID=519.topic

> > Given the
> > profile I just gave above, this generally makes it
> > easier to produce a working program that does what you
> > want.
>
>Trivially true. And good enough for many circumstances. But it is just like
>rock cocaine. It is good at first, you get a high and instant 
>gratification.
>In the long term, you are fubar. Comapre to meditation, where it takes more
>time to get the same sense of elation, but it is healthy and you can get it
>at any time -- it is part of you, not external. This is a bad analogy, but
>hopefully my point is not too obscure.

Yes.  Discipline must come from the programmer, not from
the fact that their development environment requires so
much mechanics that without discipline they will never
get anything done without discipline.  The discussion
that I linked seems doubly appropriate.

> > I firmly believe that a language may have true value
> > from just combining existing ideas in a nicer package.
> > That IMHO is what Ruby does.
>
>I anticipated this, and matz's analogy of a chef, ingredients and balance 
>is
>well taken. However it is not very interesting. Topologically, relative
>proportions of things are irrelevant.

In theory, yes.  Then again in theory theory and practice
are the same.  In practice they are not.

> > Let me give a parallel and relevant example.
> >
> > First we had Unix.  An operating system that embodied
> > the idea that it is good to have a large number of
> > simple tools which can interact easily through a
> > common medium (text delimited by returns).
>
>A nice summation. Are you familiar with Plan9? That was very influential to
>me.

By reputation but not direct experience.  As I
said, my background is math, not computers.

> > A problem with Unix is that there is a lot of overhead
> > to starting processes.  Another is that the common
> > tools that you have tend to make basic and common
> > assumptions.  If you break them you get serious trouble.
>
>Again, this is quite insightful. Unix pretends to be general and flexible,
>but in reality it is fixed and brittle. Linux is the worst thing to happen
>to computing since Java.

We will have to disagree sharply on that.  I consider
Linux to be a very, very good thing if only for creating
a platform which is challenging Microsoft's ability to
lever their monopoly into yet more markets.  I find
Linux (while far from perfect) to be more reliable both
in concept and in execution than Microsoft.

OTOH what I know about EROS suggests that it is a much
better idea than both.  Where it will go I do not know,
but check it out at http://www.eros-os.org/.  There is
not much public activity on the web page, but my
understanding is that contracts either are pending or
have been announced for specialized ATMs and kiosks.

BTW that project was written in C++ first and now is
being ported to C because of problems that they ran
into with C++...

[...]
> > I suspect that the biggest lesson which Ruby could
> > have for you is that there is a real advantage to
> > being able to program in a style where you can push
> > off accounting work to the language and concentrate
> > on your actual problem domain in small, managable
> > bytes.  (Sorry.:-)
>
>Not a good sell :). Accounting (spelling) is trivial. I am not concerned
>about that -- spell once, use many times. I am scared that there are
>concepts (compositional techniques) out there that I do not possess that
>would make my life easier.

The biggest lesson I learned from math is that simple
and easy are not particularly closely related.  Adding
10,000 numbers is a very simple task, but very few
people can do it correctly in one try.  By contrast
recognizing a voice is a very complex task, but people
find it simple and do it amazingly reliably.

Again, avoiding buffer overflows is a trivial task.
But if you leave it to the person it will not reliably
happen.  Likewise keeping track of what it means for a
given flag to be true is trivial.  Yet IIRC it is the
second most common error in studies.  Adopting a style
where flags are yes/no questions with the answer being
the value does nothing to make the task of getting it
right more trivial.  In fact to the extent that you get
longer names, your fingers have *more* trivial work to
do.  But strangely enough that change almost entirely
eliminates the error.

Perhaps this is programming domain specific, but I
really don't think so.  The less your code has to do
with the mechanics of what is going on and the more it
embodies semantic information, the more context that
our brains have and the better humans do in getting it
right.

> > >Important question: What concepts are unique to Ruby?
> >
> > Languages which are radically different from Perl on
> > the inside can reduce Perl to a class library.  And
> > there is value in that feature-set.
>
>There must be more to Ruby than just a nice wrapper for Perl. What about
>coroutines, meta-programming, genericity, interface and scope?

Yes, there is more to Ruby - a lot more - than just
being Perl with a clean design.  But your original
question (I thought) had to do with why Ruby is good
for scripting and what you could learn from that.
My answer is that Ruby is good at scripting largely
because it borrowed a proven feature-set for
scripting.

Incidentally none of the above ideas is new in Ruby.
AFAIK every one of those is intentionally borrowed
from elsewhere.

> > >I am not interested in sugar -- I want substance. Perhaps the substance
>is
> > >the language as a whole, including the developmental process it
>encourages.
> >
> > A lesson from Perl.  There is true value in syntactic
> > sugar.  Don't knock it until you try it.
>
>I take the point, although I am not convinced. += is nice, but it doesn't
>affect my design. In fact, it can add complexity where none previously
>existed (if I overload + I also must overload +=).

Funny you should mention that.  True JavaScript
example.  In Perl the .= operation is an efficient
append.  Well I was trying to produce some JavaScript
that would write tables from an internal data
structure.  So I did it like I would in Perl.

Even though JavaScript has += for this it does not do
it efficiently for strings.  As a result what should
have been O(n) became O(n*n).  Plus managed to hit
heap allocations in all browsers I tried it in.  Try
2.  Build an array and join it.  That worked much
better until I hit GC problems.  Try 3.  Build rows
as arrays, join them and append to an array of rows.
Join that when done.  That worked acceptably.

You know a decent += would have really simplified my
life.

BTW Ruby has two constructs for this.  += and <<.
I have no idea why there is a difference between
them, but for building strings incrementally be sure
to use << rather than +=.  It matters.

[...]
> > In any language, particularly ones which are easy to
> > pick up, the majority of questions you will see are from
> > beginners who are confused about basic mechanics.
>
>This is a little contradictory, and serves to illustrate my point.

Huh?

It doesn't matter how good the language is, if you
have not learned how to think about programming it
will not magically solve your problems.  In fact
judging from what I see in Ruby I predict that in
a few years lots of bad Ruby will be written just
like lots of bad VB, Java, Perl, etc have been.

Beginners will make beginner mistakes.  That will
not change in the forseeable future.  The claim is
that Ruby is easy for beginners to learn and makes
good programmers more productive.  But try to
substitute beginners for good programmers and you
will get what you deserve.  (OTOH everyone has to
start learning somewhere.)

> > Given
> > that different languages are largely used to solve the
> > same suite of problems, both problems and answers will
> > bear a remarkable similarity.  Do not mistake these
> > questions as signs of anything other than the fact that
> > Ruby is an easy language to get started with.
>
>I don't. I'm just wondering why we need to start at all. Of course, I am
>exagerating to make a point. Translation != Solution. I am also aware that
>if we have any chances of a GUT we need higher dimensions and the ability 
>to
>translate from lower dimensions to higher dimensions, a set of rules to
>apply in those higher dimensions, then a translation back to our world
>before they can be practical. What are these higher-dimensional laws that
>languages like Ruby supply? Perhaps then I could distill those laws and
>apply them to my current context without needing to make the drastic step 
>of
>learning and practically using another language. Occams Razor is very 
>sharp.

I have been repeating my best understanding of what the
general rules are.  Here are a few:

1. Make it easy to push accounting off to the language.
2. The more directly semantic content is stated, the
   better.
3. Consistency shortens learning curves and results in
   fewer gotchas.
4. Being able to put off big decisions until you have
   more experience is *good*, being able to re-evaluate
   what you are doing and rethink things is better.

And that is the point of Ruby's "everything is an
object".  It is not that everything *must* be modelled
with OO (though you can), it is that accounting work
can be pushed off to the implementations of things (1),
everything works in a consistent manner (3), it is
easy to think in terms of having lots of little
interacting components with well-defined behaviour (4),
and these three make it easy to have code that states
semantic content very cleanly (2).

While a beginner may just notice that the language is
easy to learn, an expert can extend the design
principles that make the language clean to build clean
projects on top of it.  Besides which, the language
supports enough constructs that you can write in many
different programming paradigms within Ruby.

In other words while OO ideas are embedded deep within
the design of Ruby, the option of OO is omnipresent,
but not dictated.

[...change topic...]
> > When I came to programming I noticed this thing called
> > OO programming.  As soon as I figured out what it was
> > and saw people doing it, a little light went off in my
> > brain that said, "This reminds me of category theory."
>
>It all comes down to group theory, right? At least, please correct me,
>category theory concerns itself with the translation from one ring to
>another. Another way of putting it, ignorantly, is to be concerned with the
>translation from wood to marble, or the notion of a phase transition.

I think you are mixing it up with something else.  Here
is an overview:

http://plato.stanford.edu/entries/category-theory/

Without the technobabble, the original categories of
interest were collections of objects and functions on
the objects.  Typically the objects have some sort of
internal structure and the functions between them
respect that in some way.  For instance the objects
may be groups, and the functions have to preserve
multiplication.  Or the objects may be topological
spaces and the functions need to be continuous.

The key idea of category theory is that we can forget
what the internal structure of the objects were.  We
only care about the functions.  Once you do this, it
would be a natural thing to start thinking about
taking two categories, and looking for a way to map
one category to another, send objects to objects, and
morphisms to morphisms.  All that we need to do is
make sure that the morphisms go from the right objects
to the right objects, and respect composition of
morphisms.  If we can do that then we can take real
problems in one category, turn them into equivalent
problems in another and solve it there.

Here is a small flavour.  There is a horrible
construction called homology that takes topological
spaces and gives abelian groups.  (Actually it gives a
whole sequence of them.  Yes, I know that all of the
operations are standard and apply lots of places.  It
is still horrid.)  I won't go through it.  (When you
first learn homology the experience is comparable to
building a car so you can learn to drive!)  But when
you do it the first homology group of the circle is
the integers Z, and of the disc is {0}.

If you believe that (you should, it is true), then I
can prove Brouwer's fixed point theorem.  This
theorem says that any continuous function f of the
disc to the disc sends some point back to itself.
(ie The map "has a fixed point".)

  Well suppose there was a function f from the disc
  to the disc without a fixed point.  We can define a
  function g from the disc onto its boundary by for
  each x by drawing a ray from f(x) to x, and on to
  the boundary.  This function g sends the disc to the
  circle and sends every point on the circle to itself.
  Well we can already send the circle to the disc
  (every point stays still).  So we get the following
  composition of functions (usually drawn as a triangle
  but my ASCII art isn't up to that):

   circle -id-> disc -g-> circle
          is the same as
   circle ------id------> circle

  Apply our functor above and that says you can send
  the integers to {0}, and then {0} back out to all
  of the integers at once.  Which is impossible.  So
  go back to the assumption, ah yes.  There had to be
  a fixed point after all.

  QED

Now what do integers and {0} have to do with fixed
points?  Well not much directly, but aren't they
easier to work with?  What does homology have to do
with topological spaces?  Well you always take that
step and it tends to work.  But I can't visualize
why this worked?  Stop asking for the world - we
have a proof and that should be good enough for you!

Bah humbug.

> > Indeed EVERY single person I have found who knows both
> > math and CS well enough to know what category theory
> > and OO programming are either likes both or dislikes
> > both.
>
>I am on shaky ground until you clarify my understanding of category theory.
>But it all smacks of labels: a river is something other than the word
>'river': it is wet, it flows, it has fish and it looks pretty in the
>morning.

Hopefully the above helps.  Basically category theory comes
down to a set of formal constructions that let you turn
unmanagable problems about complex things like rivers which
are wet, flow, suffer from pollution, etc into nice simple
set-piece problems which we have a ton of machinery to
handle.

Pretty is a distraction. :-)

Reducing problems to well-understood patterns immediately
clues in people who are familiar with them.  And if you
always see the world that way, it is much more efficient
than having to figure out what someone is doing because
they are not using the familiar construct (but they are
probably doing something familiar).

>To say that I have reservations about OO is an understatement. I've seen
>(and built) too many rigid, awkward, inappropriate, and limiting class
>hierarchies to think otherwise. The notion of a virtual method is required
>but is very very much overused. Has-a is stronger than is-a.

Absolutely agreed.  OO is a very unrealistic way to
model the world.  Try to shoehorn complex reality
into an elaborate OO design and the conflict shows
up in broken and fragile code.  Happens time and
again.

Generally containment through has-a is a much cleaner
design.  Is-a implies among other things that you
have the same implementation, which you often do not
want.  What is more useful is to inherit the wanted
aspects of the *interface*.  And inheriting multiple
interfaces is not nearly as bad as trying to manage
inheriting multiple implementations.  A point that
lies behind why it is saner to inherit from multiple
modules than classes.

A related point is that typically what something
*does* matters more than what it *is*.  And that is
what Ruby's dynamic typing handles.

> > Unsurprisingly, I find that I am quite capable of doing
> > OO.  I understand exactly why it is useful.  But when
> > you begin talking about things like design patterns it
> > feels like meaningless symbol manipulation and I don't
> > enjoy the process very much.
>
>A man after my own heart :). Patterns are for ignorant prats that can't
>think for themselves and need some artificial formalism in order to supply
>some borrowed sense of rigor and methodology.

Um, take a look at the object-oriented design modules in
Ruby.  In Ruby through dynamic typing it is very easy to
implement the standard OO design patterns generically.

In principle while you can work out how to do these
things from scratch every time, why?

>Patterns are funny.

It is hard for me to find them interesting either,  But
enough people whose opinions I respect find them
fascinating that I have concluded that they really do
help people organize their thoughts and spot familiar
constructs in new situations.

[...]

Cheers,
Ben
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com