Adventures in Flatland: Perceiving Social Interactions Under Physical Dynamics