I've written a small blog about the fall of the Monkey Island series over at The Monkeys Are Listening
and the reason is very similar to that of Grim Fandango. At the end of the 90s it was obvious that the technology was advancing so far to create more 'interesting' games such as shoot-em-ups. Adventure Games, despite great plot and characters, just didn't stand up to the action of shoot-em-ups. Puzzles paled compared to blasting the hell out of some alien threat. LucasArts, like many gaming companies, decided to jump ship and create these action games but use their main asset, being license to Star Wars and create Star Wars action-games. Of course they had made Star Wars games before even SMI was released, but now the better technology was there they could create a apparent 'better Star Wars' experience.
In short, Star Wars sells; it always has, but Adventure Games such as Grim Fandango and Monkey Island fail to meet those sales. Yet, to add a good note, both of those games have far more content and character than any of their Star Wars games; why? Because they're original concepts. Well almost original, yet at least they are not the same old Star Wars rehashed under new titles.