We introduce a general characterization of sudden cosmological singularities and investigate the classical stability of homogeneous and isotropic cosmological solutions of all curvatures containing these singularities to small scalar, vector, and tensor perturbations using gauge invariant perturbation theory. We establish that sudden singularities at which the scale factor, expansion rate, and density are finite are stable except for a set of special parameter values. We also apply our analysis to the stability of Big Rip singularities and find the conditions for their stability against small scalar, vector, and tensor perturbations.