The overwhelming popularity of the Smeeta Kavat is undeniable, and this popularity can be traced back to three factors:
A) Humans are bad at statistics and randomness - The Charm buff often gets read as "I get all of these different buffs." Even though we know that it's only a chance every x minutes, humans have a hard time internalizing this. Since one of them is always appealing, people tend to see Smeeta as never being a bad choice and usually see it as a good choice.
B) Inertia - Players start using Smeeta early and, because it survives and has some usefulness, they stay with it. The Smeeta Kavat has decent health and a bleedout timer making it viable early through late-game. It became part of the meta and continues to be viable. This means the early investment of resources and time to learn Smeeta pays larger dividends because Smeeta is never a bad choice.
C) Other Companions are not as good - Many companions just can't survive a whole mission especially long, high-level content. The buffs from most other survivable companions are only useful for certain builds or play styles while Smeeta is never a bad choice.
Fixing companions diversity is simple. It requires fixing survivabilty and increasing the usefulness of all companions. (I'm not so ambitious as to try and explain, or even argue about, the statistics of Charm.)
1) Give all companions a bleed out timer. (And make the Regen Mods kick in after the timer runs out.)
2) Make a (consumable) Gear item to revive a dead companion mid-mission. (Yes, in addition to 1.)
3) Make all companion ability and stat Mods scale off of something in addition to Mod Rank. For stat mods, this would be like the Link Health/Shields/Armor Mods. For ability Mods, it would scale off a Warframe or weapon stat - e.g. Shields, Ability Range, Melee Weapon Damage, etc. - depending on what was appropriate for that ability. There will be some situations where specific abilities are really good, but this is exactly what we want. Different situations or strategies where each option is really good. It will be based on Warframe selection and the build making it highly unlikely that there would be any "one-size-fits-all" optimal build.
This would create so many ways to tweak builds and factors to consider when choosing Companions and abilities based on which Frame and build you are using, that we would have many more viable and useful companions and would see a much wider array of companions in missions.