© 2020 American Physical Society.Despite its importance to experiments, numerical simulations, and the development of theoretical models, self-averaging in many-body quantum systems out of equilibrium remains underinvestigated. Usually, in the chaotic regime, self-averaging is taken for granted. The numerical and analytical results presented here force us to rethink these expectations. They demonstrate that self-averaging properties depend on the quantity and also on the time scale considered. We show analytically that the survival probability in chaotic systems is not self-averaging at any time scale, even when evolved under full random matrices. We also analyze the participation ratio, Rényi entropies, the spin autocorrelation function from experiments with cold atoms, and the connected spin-spin correlation function from experiments with ion traps. We find that self-averaging holds at short times for the quantities that are local in space, while at long times, self-averaging applies for quantities that are local in time. Various behaviors are revealed at intermediate time scales.