it may be possible that key retrival has, like the key stealing itself, a probability. keep attacking and see if you get it with 10 attacks in a row on that planet. (it doesn't matter how much units you send)
To steal a key from a player you must have lower score points from him. No matter how many times you attack, if you have higher points you will never "steal" back your key. If it is not used in any other way, it will return to you in 3 days.