It is possible that you have high-frequency hearing loss. With this type of hearing loss, you can hear well in one-on-one situations, and even in small groups. However, when you are around distracting speech and noise, you can hear the noise louder than the speech.
Your low-frequency hearing picks up the low-pitched noise at a normal hearing level. This causes you to miss some of the high-frequency speech sounds, which is where your hearing loss is located. This hearing loss is not noticeable when speaking with someone without any competing noise.