First off, let's not forget that the stat modifier is not a modifier to the stat bonus, but to the stat itself. This means that Elves can have a maximum stat of 17 for Willpower, while Dwarves can have a 20.
2 characters with the same final stat have the exact same bonus, so that -3 won't come into play there.
Now, I would be willing to change the enhanced senses to a affinity to magic (change the +2 from Perception to Spellcraft), or to split that +2 modifier between the two (+1 to Perception and to Spellcraft).
That alone would make elves be "more magical" in nature because then with equal stats and skill ranks, the elf would be better. Then again, why should ALL elves be better at magic?
If a player wants to be "a magical elf", he can always take the Natural Ability talent for Spellcraft.