Inflation is a phase of accelerated expansion that takes place at very high energy in the early universe. During this epoch, vacuum quantum fluctuations are amplified to seed the large-scale structure of our universe. When inhomogeneities are produced with sufficiently large amplitude, they may subsequently collapse into primordial black holes. I will explain why the effect of quantum diffusion during inflation needs to be taken account in such a case, and how the abundance of primordial black holes can be predicted while consistently accounting for such effects. I will show that the result can vastly differ from standard calculations, and discuss consequences for inflation and for primordial black holes.