"Me neither" in almost all situations is being used to mean "I don't want to". Therefore, the word "me" is being used as the subject of the sentence and is incorrect, because "me" is the objective case.
I know that "me neither" sounds normal because it is commonly used. However, the question concerns what is correct English, so any of us who have come to this site recognize that there is a standard to which we comparing our speech.
Therefore, the concept that if a "native speaker uses it, then it must be correct" is abhorrent to me. If that were true, there would never be English classes in school and there would be no sites like this!