Adiabatic \(U(2)\) geometric phases are studied for arbitrary quantum systems with a three-dimensional Hilbert space. Necessary and sufficient conditions for the occurrence of the non-Abelian geometrical phases are obtained without actually solving the full eigenvalue problem for the instantaneous Hamiltonian. The parameter space of such systems which has the structure of \(\xC P^2\) is explicitly constructed. The results of this article are applicable for arbitrary multipole interaction Hamiltonians \(H=Q^{i_1,\cdots i_n}J_{i_1}\cdots J_{i_n}\) and their linear combinations for spin \(j=1\) systems. In particular it is shown that the nuclear quadrupole Hamiltonian \(H=Q^{ij}J_iJ_j\) does actually lead to non-Abelian geometric phases for \(j=1\). This system, being bosonic, is time-reversal-invariant. Therefore it cannot support Abelian adiabatic geometrical phases.