Conductance fluctuations in GaAs quantum dots with spin-orbit and Zeeman coupling are investigated experimentally and compared to a random matrix theory formulation that defines a number of regimes of spin symmetry depending on experimental parameters. Accounting for orbital coupling of the in-plane magnetic field, which can break time-reversal symmetry, yields excellent overall agreement between experiment and theory.