Imagine that one did have a mental image of the facial expression that one is now perceiving, and suppose that the expression and one’s image coincide. But, it can be asked: how does one know that the relation between the facial expression and the image is one of coincidence or of agreement? We would still need to recognize the relation between the expression and the image as one of coincidence. But, in that case, we need to have a mental image of the coincidence between the image and the expression, and to interpret that the image of the coincidence coincided with the coincidence! T.S. Elliot captures this muddle of thought in the following quip:

Footfalls echo in the memory
Down the passage which we did not take
Towards the door we never opened

One never knows when one is making a memory. At least, one can never know if a previous memory is taken to confirm it. The point is that a mental image of your facial expression cannot explain my recognition of it, for I would have to recognize that the image is an image of the expression. But the philosopher wants to insist that

‘ “This object is familiar to me” is like saying “this object is portrayed in my catalogue”.’ In that case it would consist in the fact that it was a picture filed with others in a particular folder, in this drawer. But if that really is what I imagine – if I think I simply compare the seen object with pictures in my catalogue and find it to agree with one of them – it is something quite unlike the phenomenon of familiarity. That is, we are making the assumption that the picture in our catalogue is itself familiar. If it were something strange, then the fact that it was in this folder, in this drawer, would mean nothing to us. (Wittgenstein, Philosophical Grammar, §179f.)

It is quite imaginable that one may note a facial expression on a person, acquire the memory image that coincides with one’s current impression of it, and yet not remember who one’s memory-image is an image of, and therefore, not recognize the facial expression on the face before one. Unless one knows what a memory-image is an image of, it is quite useless for recognition. It takes memory to tell me whether what I experience is the past. In any event, if one knows what it is an image of, why then do we need it in the first place?

The philosophers maintain that when I recognize your facial expression, I fit my current impression of expression x with a mental image derived from a previous experience of x, like a piston fitting into a cylinder. What, then, of Plato’s metaphor of the wax tablets? Do we know of any such imprint in our experience of recognition? Again, even if there were an imprint, how would we know whether the current perceptual impression fits it? Russell said that the coinciding between my impression and memory-image is told by a special feeling of coinciding. But, how would we recognize the feeling? Suppose now that the imprint signals to us in facial recognition. Conceivably, I might be able to coordinate my present impression and memory-image for a fit; but again, it is equally conceivable that I not yet recognize the face, because I do not remember whom the imprint is of. Wittgenstein writes:

        When I speak of a pattern in my mental catalogue, or of a sheath into which an object fits if it is familiar, what I would like to say is that the sheath in my mind is, as it were, the ‘form of imagining’, so that it isn’t possible for me to say of a pattern that it is in my mind unless it is really there. – The pattern as it were retires into my mind, so that it is no longer presented to it as an object. But that only means: it didn’t make sense to talk of a pattern at all. (The spatial spectacles we can’t take off.)

           If we represent familiarity as an object’s fitting into a sheath, that’s not quite the same as our comparing what is seen with a copy. What we really have in mind is the feeling when the object slips smoothly into the contour of the sheath. But that is a feeling we might even if there were no such perfectly fitting sheath there at all.

           We might also imagine that every object had an invisible sheath; that alters nothing in our experience, it is an empty form of representation. (Wittgenstein, Philosophical Grammar, §180)

Wittgenstein concludes that the philosophies of recognition and memory are flawed because they misrepresent and misconceive recognition and memory. A good starting point for the philosophers is to convey ordinary experience as what it is, and not according to some preconceived schema.