My lovebird panics when I put my hand in the cage?

You have to understand that lovebirds are more reserved birds and opposed to parrots and larger birds, they are less social with any other kind of animal. They rarely get along with other birds! Don't feel bad about it being frightened by your hand. The fact that it happily hops on your finger is a huge achievement! By nature, they don't usually associate themselves with any other creatures so it's not your fault. Just don't force things onto it and I'm sure in time it will come to be more than comfortable with you.