I'd say how little they got out of Monta Ellis was a big factor. His lack of contribution offensively made their offense too limited. I expect that was seen as a failure of Vogel's given Monta's consistent offensive output elsewhere in his career and given that this was an offensive system which Monta should have done well in.
The Pacers never really got to the free-wheeling offense that Vogel was supposed to build. So in that regard, I can see where Bird is coming from. Vogel failed to take the team far enough into the direction Bird wanted them to go. And Vogel failed to get the best out of his 2nd best offensive threat. The Pacers did well in regular season and playoffs but not in the way that Bird had intended (more about their defense than opening up the offense) - the direction he wanted the team to move towards. I don't think Vogel was fired due to a lack of success (45 wins, 7 game 1st round series loss) but due to the style of the team. That Vogel failed to create the type of offensive system that Bird wanted / envisioned.
I am looking forward to seeing the new-look Pacers next season. I think they can do a lot more offensively with a new coach + the continued growth of Myles Turner.