It's a good problem to have. The Yankees sure aren't complaining and they have as many retired as the C's (plus they have 40 players per season and a manager with numbers). The teams with the most retired numbers unsurprisingly are amongst the best franchises in their respective sports.
If we keep retiring numbers, that's a good thing. That means we are having quality players making significant impacts on our franchise.
I hope we run out of numbers to use.