If your hero's rank is upgraded, the hero's leadership also increases which enables the hero to have increased limit of marchable army.

If your hero's grade is upgraded, skill level and effect will be increased.

Also, certain army skill and hero skill are unlocked at certain ranks.

And the effect increases with the hero’s grade.

Each hero is required to be equipped specific Runes on slots for hero's rank upgrade. 

For hero's grade upgrade, you need to collect Soulstones required depending on grade of each hero.