The ability of graphene to support long-lived, electrically tunable plasmons that interact strongly with light, combined with its highly nonlinear optical response, has generated great expectations for application of the atomically thin material to nanophotonic devices. These expectations are mainly reinforced by classical analyses performed using the response derived from extended graphene, neglecting finite-size and nonlocal effects that become important when the carbon layer is structured on the nanometer scale in actual device designs. Here we show that finite-size effects produce large contributions that increase the nonlinear response of nanostructured graphene to significantly higher levels than those predicted by classical theories. We base our analysis on a quantum-mechanical description of graphene using tight-binding electronic states combined with the random-phase approximation. While classical and quantum descriptions agree well for the linear response when either the plasmon energy is below the Fermi energy or the size of the structure exceeds a few tens of nanometers, this is not always the case for the nonlinear response, and in particular, third-order Kerr-type nonlinearities are generally underestimated by the classical theory. Our results reveal the complex quantum nature of the optical response in nanostructured graphene, while further supporting the exceptional potential of this material for nonlinear nanophotonic devices.