Note: This answer assumes that Rescue Energy is being activated by the Knock Out.
For that to happen, Gengar has to be using an attack that does damage.
His normal attack doesn't. It places damage counters! So that won't activate Rescue Energy.
If you have a Gengar Level X on it, and use that attack, which does do damage, then the choice exists.
If you are KO'ing Gyarados via an attack which places damage counters, then Rescue Energy does not activate and Gyarados and the Magikarp go to the Lost Zone.