How 'bout rubber cement? (Course, I don’t know what that would do to extruded foam - if that is what you might be attaching it to.) Another suggestion is Hob-e-tac. Make sure you let it dry thoroughly so that it becomes tacky.
I’ve used glue, but what I like better is drilling a small hole into one of the feet and inserting a piece of paper clip until it extends about 1/4" It stays put and I can move it at a whim.
There used to be on the market a little container of special wax,I suppose beeswax would work, A tiny ball of this wax under the figure will hold it in place and make it removable later. Sorry,I forgot the source.I’m afraid it might be called embalmers wax.r
Rockler’s Woodworking stores sell a VERY thick CA glue with a spray bottle of accelerator. A tiny dab of CA on the feet, stick the figure in place with one hand and squirt with the other. Instant stick, no falling over.
Mostly, I use Aileens Tacky Glue, a craft-store product. It holds fine, but you can still get things up and easily scrape off the glue. It doesn’t set up instantly, so you have to support the figures until it does or they’ll fall over.
Sometimes, I just use CA if I’ve got limited contact surface. To get those figures off, I’d use a knife or razor blade.
I cut 3/8 inch squares from an old clear report cover (the kind you use in school) and glue them to the feet of figures with CA. I can move them any where I want, any time with no residue left behind. The plastic is stiff enough in this size to hold the figure upright, but clear and thin enough to be fairly unobtrusive.
It’s called snot glue because it texture, and sometimes color, are reminiscent of snot. I’m not sure where to find it anymore - probably a good crafts store. Just ask one of the clerks for snot glue.