Arrays of Rydberg atoms in the blockade regime realize a wealth of strongly correlated quantum physics, but theoretical analysis beyond the chain is rather difficult. Here we study a tractable model of Rydberg-blockade atoms on the square ladder with a \(\mathbb{Z}_2 \times \mathbb{Z}_2\) symmetry and at most one excited atom per square. We find \(D_4\), \(\mathbb{Z}_2\) and \(\mathbb{Z}_3\) density-wave phases separated by critical and first-order quantum phase transitions. A non-invertible remnant of \(U(1)\) symmetry applies to our full three-parameter space of couplings, and its presence results in a larger critical region as well as two distinct \(\mathbb{Z}_3\)-broken phases. Along an integrable line of couplings, the model exhibits a self-duality that is spontaneously broken along a first-order transition. Aided by numerical results, perturbation theory and conformal field theory, we also find critical Ising\(^2\) and three-state Potts transitions, and provide good evidence that the latter can be chiral.