Under the assumptions of diffuse wavefields or energy equipartitioning, theoretical studies showed that the Green’s function can be retrieved from the cross-correlation of ambient noise in seismic interferometry (SI). However, in practice, correlograms are not equal to the empirical Green’s function since the assumptions for correlation-based SI are generally not satisfied in realistic situations. In the framework of supervised learning, we propose to train deep neural networks to overcome two limitations of correlation-based SI: the temporal limitation of passive recordings, and the spatial limitation of the random source distribution. Deep neural networks are trained to implicitly find the relationship between the empirical Green’s function and the correlograms, and then used to extract the correct Green’s function from ambient noise. The input of the network is correlograms (a virtual shot gather) and the desired output is the empirical Green’s function (the active shot gather). Numerical examples show that a deep network aware of the source directionality (through a preliminary beamforming step) can help mitigate some of the challenges associated with inhomogeneous source distributions. In this work, all the numerical examples are based on the retrieval of P-wave reflections at exploration scales, and are conducted on synthetic data. Many precautions are taken to avoid the “learning crime” where the training and testing scenarios are too closely related. We use the CycleGAN architecture in all our numerical experiments.