Axel Etzold wrote on Thu, 19 Jun 2008 20:33:43 +0900

> well, what does similarity of two images/textures mean for you ?

Perceptual similarity as a human subject would experience it. Let me 
expand (=ramble) on this:

At the moment I am focusing on following the problem: Given any single 
photograph and a random set of photos (20-100), which of the random set 
is most similar, perceptually, to the target photo.

I have made some simple tests that divide image into color-channel 
components and a luminosity channel, downsample the channels into a 
16x16 arrays, and calculates the difference between the target photo to 
each of the random ones. Difference hashing its called?

Results are rather confusing. Most of the time perceived similarity (as 
I experience it) does not exist, even if statistically the images might 
be similar.

IOW, random bw-noise-image gets rather good similarity scores whilst 
morphologically having zero resemblance to the target photo.

But anyone who has studied this subject even superficially already knows 
this - hashing wont produce the same kind of similarity we experience 
with our senses.

> Given a (hopefully large) set of images, you could divide it into a 
> training set and a set of images to be classified into sets of mutually
> similar images. One way to perform both the training and the classification
> is using Support Vector Machines. I found these Ruby bindings to a library
> providing these by a Google search, haven't used them yet :
> 
> http://sourceforge.net/projects/rubysvm/

This is one of the possible avenues. Gabor features sets used this kind 
of approach I believe. Thank you for that link.

But, I don't see this as the most interesting approach. The particular 
problem I wrestle with has rather small sets, and training would have to 
be performed for every photo. Not really practical or efficient. May be 
training a nn is the only way to really do this.

> Most probably, you'll need to read out image information at the pixel level at some point.
> Imagemagick is a very powerful library to do this, and Tim Hunter provides
> a wonderfully rich Ruby binding to it: RMagick.

I have been using RMagick, AND have been looking at Hornets Eye, which 
might be useful at some point, but at the moment doesn't cater to this 
specific case.

> Psychologists who have conducted image similarity studies compare many different
> measures and filtering methods,e.g., here:
> 
> @misc{ karl-perception,
>   author = "Dirk Neumann Karl",
>   title = "Perception Based Image Retrieval",
>   url = "citeseer.ist.psu.edu/635475.html" },

Too bad that service isnt available, as I have not read that article.

> At some point, most of these methods use a (discrete) Fourier transform of some of the
> image information and compare the results of the transforms of two images to assess their
> similarity.
> You could use Ruby-GSL, or Narray with fftw3 to perform that.

Narray is what I was planning to use once theres a working model for the 
similarity comparison in place.

E. Borasky - Yes, I have the tools down, but clearly dont have a 
suitable perceptual image comparison algo yet.

Thanks also to Ron Fox for pointing out its not going to be easy. :)

So, I guess I could use the rest of my lunch break to elaborate on the 
Question.

What kind of computational algorithm would provide the perceptual 
similarity score, rating or hash of some kind between two or more images 
that would match the way humans perceive best?

I guess one would need two distinct classifications: similarity of 
morphological appearance (features, shapes, ? in image) and similarity 
of the colors (of the areas).

I assume there is no answer in existance to this question, well, that I 
know of.