I think that you have been correct. I would see the endearment as a substitute for the person's name and as such should be capitalized--much like one capitalizes Mother, but not my mother.