Paul, I’m not sure if this would solve your problem or make it worse…
bragdon enterprises has weathering powders with adhesive in them. They stay put so that’s the good/bad news, but you’d likely have no “dregs” sitting on the base afterwards. You have to be more careful than with chalks, etc. because what you put on your model is very hard to remove. Just thought I’d mention this product in case you hadn’t heard of it.
The owner of the company has the best rock molds in the hobby too and is VERY helpful via emails or phone calls.
Tim mentions the use of Braegdon weathing powders, this is what I use. They have an excellent adhesion. When using the powders, I do the weathering over Sheet plastic or printed cardboard, where the excess is easy to pick up w/ the brush or “make-up” pads, many times I will use the surface as a “pallet” w/ differing colors, mixing in the center of the ring of shades. The small piles can be brushed back into the bags, but many times I will just leave the powders covered for an additional use. This is usually done when doing mutiple cars of similar weathering. For a simple "quick"application, small plastic bags or a block of wood will suffice. Any excess that falls on the work/ powder suface is reused or saved.
try wetting your brush with alcohol and picking it up. Many times I mix the powder with alcohol and make a wash, once the alcohol evaporates the powder is adhered pretty good