I believe the reason is that the higher handicapper has much more room for improvement than the lower handicapper does.
If a 28 handicapper improves by 10% then he is 3 under par
If a 5 handicapper improves by 10% then he is under and just lost the match 2 down.

The other side of the coin is that when you play of fractions the higher handicapper gets crucified and the lower one doesnt.
e.g 3/4 of 28 is 21
3/4 of 4 is 3
so the high guy loses 7 shots, the low guy loses 1

to me this only makes sense in team events when u have multiple chances at scoring to balance out the shot loss.