Reconstructing deformable objects from video requires a 4D representation that should preserve geometry, explain motion through physics, and generalize to future or new physical conditions. Existing dynamic NeRF and Gaussian methods achieve strong view synthesis, yet their voxel or ellipsoidal primitives are primarily designed for appearance rendering and typically driven by centers or predefined particle bindings, limiting physically meaningful non-uniform deformation and sharp boundary evolution. We present PhysConvex, a physics-informed dynamic convex field for video-based reconstruction, physical system identification, and simulation. PhysConvex represents a dynamic object as material-space deformable convex primitives whose boundary is advected by physical dynamics. The boundary-driven convex expresses non-uniform deformation and evolving active supports while serving simultaneously as a rendering element, deformation carrier, and physical support where mass, elastic response, forces, and contacts are evaluated. We further introduce a mesh-free reduced-order convex simulator in which neural skinning modes define physics-based deformation bases directly over deformable convex supports. Experiments show improved dynamic and physical reconstruction, efficiency, future prediction, and generalization to changed materials, forces, and boundary conditions.