Bird Issues....?

My bird we got a few weeks ago. We have a cat, but he is too lazy to be getting at the bird. Anyway I walk by the bird cage and he always hisses at me. It is starting to get annoying because I try to be nice and he pecks at me. Picking him up and letting him out of his cage I have tried several times (I locked up the cat in case) and he won't ever come down again. I have to chase him with a broom. He's a atiel. Major bird problems, help!