The amount of energy in the noise and the damage it can do to your hearing increases very rapidly as it gets louder. In fact, noise energy doubles for every 3 decibels (3dB) increase in the loudness of the sound - and 3dB is such a small increase in loudness that you probably wouldn't even notice it.
1More