Camera Calibration: very large k3 value in distCoeffs

edit

I run cv2.calibrateCamera on 67 images with 19x3 chessboard corner detections in each. I find reprojection error around 0.3. When I undistort images, they look great. They had little distortion before, and appear to have basically no distortion afterward.

Later, I use these camera calibrations (and those from the other camera in a stereo setup) as input to stereoCalibrate, which returns error about 0.3. When I try to stereorectify and remap, I get nonsense.

This led me back to the original camera matrices. My distortion coefficients from the camera calibration are

array([[-1.44561083e-01, -3.21620254e+01,  0.00000000e+00,
     0.00000000e+00,  1.72034253e+03]])

K3 seems very large! Any idea what is going on here? Could this be ruining my stereo rectification later on?