Brain tumors can cause direct damage to the nerve, or indirect by disrupting the flow of blood, thus starving the nerve. This damage may happen even if the tumor is not growing. The other cause for damage is the so called "tumor related edema", i.e. inflammation of the surrounding structures.
The doctors know that there is a big possibility for these to occur while watching and waiting, but they also know that you may not be better off after removing or radiating the tumor, as the hearing loss may happen anyway.
Very often hearing loss can be reversed, if caught early. Fluctuations in hearing and distortion are not uncommon either. When I experienced by first sudden hearing loss a month after I was diagnosed, I was kind of resigned to it, as in "oh well I have an AN, this is expected". My husband was the one who convinced me to contact the doctor right away. Sure enough, the doctor prescribed steroids and hearing was saved (at least for now
Recently I felt some "fullness" and some slight hearing loss (I have my own hearing test by asking hubby to speak to me on the phone - his frequencies are the ones that I have most difficulty with). I took 400 mgs of ibuprophen 3 times a day for 3 days, and the symptoms resolved.
I would say that you should contact your doctor ASAP if your symptoms continue.