walkaround throttle question

I recently added an MRC Controlmaster 20 to my layout. I have elected to wire a network of phone jacks around the fascia so that the CM20 handheld can be moved from point to point around the room to follow a train. The only problem I’ve experienced is that whenever the handheld is disconnected to move to the next jack, there is a droppage in power. It is just enough to be aggravating…trains at cruising speed slow notably, but anything running slowly to start with often stops completely until the handheld is plugged back in. The handheld has a memory and is intended to be used in this manner, but surely without the slowdown I’m experiencing. The problem is just as bad at the first jack near the power unit as it is down the line further. As near as I can tell all the wires in each jack are connected well. Has anyone else with a CM20 had this problem?