What could be wrong

I have a 2002 Buick regal ls, when I try to start it,
it just clicks once, the lights on dashboard go off
and headlights too, as soon as I let go
everything comes back on, headlights are bright
again, I thought it was the starter and battery,
took both off and took it to autozone they said
starter is good and I took battery and put it on
another car and it starts right up,  I though
Mabey it was corroded so I bought a wire brush
and cleaned both,  still just clicks once, help me
please.