According to arstechnica.com
"Most of the time, when you talk to an Amazon Echo device, only Amazon's voice-recognition software is listening. But sometimes,Bloomberg reports, a copy of the audio is sent to a human reviewer at one of several Amazon offices around the world. The human listens to the audio clip, transcribes it, and adds annotations to help Amazon's algorithms get better."