The thickness-dependent third-order nonlinear optical properties of two-dimensional β-InSe and its potential applications as a saturable absorber in pulsed laser generation are investigated. InSe sheets with different layers are prepared by the chemical vapor deposition. Using open-aperture femtosecond Z-scan technique at 1030 nm, the modulation depth and nonlinear absorption coefficient are obtained to be 36% and -1.6 × 104 cm·GW-1, respectively. The intrinsic mechanism of the layer-dependent energy band structure evolution is analyzed based on density functional theory, and the theoretical analysis is consistent with the experimental results. Based on a waveguide cavity, a Q-switched mode-locked laser at 1 µm with a repetition frequency of 8.51 GHz and a pulse duration of 28 ps is achieved by utilizing the layered InSe as a saturable absorber. This work provides an in-depth understanding of layer-dependent properties of InSe and extends its applications in laser technology for compact light devices.