Acoustic signatures can be used in connection with a voice-enabled computer system. An acoustic signature can be a specific noise pattern (or other sound) that is played while the user is speaking and that is mixed in the acoustic channel with the user's speech. The microphone of the voice-enabled computer system can capture, as recorded audio, a mix of the acoustic signature and the user's voice. The voice-enabled computer system can analyze the recorded audio (locally or at a backend server) to verify that the expected acoustic signature is present and/or that no previous acoustic signature is present.