Austrian economists say this all the time (including Saifedean @saifedean ): “Economics is not physics because there are no units, people are not particles,” and so on.

But this misunderstands both economics and physics.
You can build an entire science based on relationships, scaling laws, and invariances without relying on simple physical units. In fact, some of the deepest ideas in physics are based on dimensionless ratios and symmetries rather than meters or kilograms.
And while people are not particles, modern physics is not just the study of particles. Physics studies complex systems made of many interacting components. The same mathematical and statistical tools developed in physics have been successfully applied to biology, ecosystems, traffic flow, neural networks, epidemics, cities, and social systems.
Econophysics and network science exist precisely because large collections of interacting agents often display emergent regularities independent of the microscopic details of the individuals involved.
The objection usually comes from an outdated view of physics, one rooted in the 1800s mechanical picture of billiard balls colliding in space. But physics evolved far beyond that long ago.
Modern physics is fundamentally the study of patterns, interactions, scaling behavior, information flow, and emergent order in complex systems. Economists, including many Austrian economists, often criticize a caricature of physics rather than the real discipline as it exists today.