A candidate accreting massive black hole (BH) with M_BH ~ 10^6 Msun has recently been identified at the center of the dwarf starburst galaxy Henize 2-10 (He 2-10). This discovery offers the first possibility of studying a growing BH in a nearby galaxy resembling those in the earlier universe, and opens up a new class of host galaxies to search for the smallest supermassive BHs. Here we present very long baseline interferometry observations of He 2-10 taken with the Long Baseline Array (LBA) at 1.4 GHz with an angular resolution of ~0.1" x 0.03". A single compact radio source is detected at the precise location of the putative low-luminosity active galactic nucleus. The physical size of the nuclear radio emission is < 3 pc x 1 pc, an order of magnitude smaller than previous constraints from the Very Large Array (VLA), and the brightness temperature of T_B > 3 x 10^5 K confirms a non-thermal origin. These LBA observations indicate that the nuclear radio emission originates from a single object and exclude the possibility of multiple supernova remnants as the origin of the nuclear radio emission previously detected with the VLA at lower resolution. A weaker, more extended, off-nuclear source is also detected with the LBA and a comparison with multi-wavelength ancillary data indicate that, unlike the nuclear source, the off-nuclear source is co-spatial with a super star cluster, lacks a detectable X-ray point-source counterpart, and is almost certainly due to a supernova remnant in the host star cluster.