It’s more than just signalling. There are other engineering issues. For example, the degree of curve banking required for high speed passenger trains may be inconsistent with operation of modern, high center of gravity freight cars. Also, there’s a difference in design philosophy. A freight railroad will typically be designed to minimize grades as much as reasonably possible, and this often involves increasing curvature to reduce grades. Grades aren’t as big a problem for high speed rail, but curvature is, because curvature limits maximum speed The " grade vs curvature" issue won’t be that evident on lines with relative flat terrain, but it would be very important in hilly or mountainous terrain.
There’s also a maintenance issue. High speed passenger service requires that track be maintained to very tight tolerances. But operating heavy freight trains on the same tracks makes it difficult if not impossible to keep the track structure within those tolerances. Anyone who’s ridden an Amtrak train over trackage which sees heavy coal traffic has experienced this problem first hand