We study the smoothness of the black hole horizon in the Hayden-Preskill thought experiment by using two particular toy models based on variants of Haar random unitary. The first toy model corresponds to the case where the coarse-grained entropy of a black hole is larger than its entanglement entropy. We find that, while the outgoing mode and the remaining black hole are entangled, the Hayden-Preskill recovery cannot be performed. The second toy model corresponds to the case where the system consists of low energy soft modes and high energy heavy modes. We find that the Hayden-Preskill recovery protocol can be carried out via soft modes whereas heavy modes give rise to classical correlations between the outgoing mode and the remaining black hole. We also point out that the procedure of constructing the mirrors of the outgoing soft mode operators can be interpreted as the Hayden-Preskill recovery, and as such, the known recovery protocol enables us to explicitly write down the mirror operators. Hence, while the infalling mode needs to be described jointly by the remaining black hole and the early radiation in our toy model, adding a few extra qubits from the early radiation is sufficient to reconstruct the mirror operators.