We report the observation of the sign reversal of the magnetic correlation from antiferromagnetic to ferromagnetic in a dissipative Fermi-Hubbard system, utilizing the dissipation caused by on-site two-body losses in a controlled manner. We systematically measure dynamics of the nearest-neighbor spin correlation in a double-well optical lattice, as well as a crossover of a double-well lattice to a one-dimensional uniform lattice. In a wide range of lattice configurations over a double-well lattice, we observe a ferromagnetic spin correlation, which is interpreted as a transient state leading to a Dicke state. This work demonstrates the control of quantum magnetism in open quantum systems with dissipation.