Should I let my hair go gray?

I'm 46, but everyone tells me I look only 30. This is not just flattery. I really do only look 30, but my hair is turning gray. I color it to my natural very dark, almost black, brown, and it looks very natural and y. I'd like to stop coloring it, but I wonder if it would be too strange considring my younger look, or would it make me suddenly look much older. I would have about 25% gray at this point but no wrinkles...excellent skin, no need for foundation. So shoudl I keep coloring or go natural? My hair is quite long and VERY, wildly curly. I get lots of compliments, and I'm told my hair gives me an exotic look, but sometimes I think looking so much younger makes me a fraud...as well as making the men I am attracted to (50-55 years olds) think I look too young, since the more mature ones don't like looking foolish being with a much younger woman, and a 55 year old dating a 30 year old (even if she's really 46) can look foolish.