As you suspected, "me too" is grammatically incorrect as "me" is in the accusative case. Technically, you are saying the persons wants to go to store and you, or has you as much as they have ten fingers. However, like all applications of language, culture and context can render such phrases as appropriate.