Apart from room acoustics etc. the microphones have a too strong self-sound, to really allow valid results. I had tested more than half dozen different handheld recorders, from Olympus to Yamaha and Zoom. And all of them sounded so different that I doubt they would allow any neutral results.
To give it another try, I lately ordered a Zoom H5 and an optional XYH-6 capsule as I hoped these would give me significant improvements over the "cheap" Zoom H1 recorder that I am currently using, but in the end, the results were even worse. Not only did both mic-capsules emphasize treble too much, they did roll-off with low frequencies too strong, and there was a noticable divergency between left/right channel with the sound being not centered as it should be. Both microphones had a different bias towards left or right, therefore I sent both back totally disappointed.
Currently it seems as if I was still at the very beginning to find a better but still affordable recording solution which would allow the most neutral and flat recording.
You can hear some of my previous test-recordings here. In this case the JBL Charge 2 was recorded with all recorder/microphone combinations in the same room with the same settings/distance etc. As the XYH-6 capsule supports 2 different directional patterns, I did recordings with both, you can also check out the original track for comparison:
Excerpt of the original track:
Zoom H1:
Zoom H5:
Zoom H5+XYH-6 90°:
Zoom H5+XYH-6 120°:
I think it is pretty obvious that all the used microphones have a different sound, there is even a noticeable sound-difference between the 90° and 120° setting of the XYH-6 capulse. I also tried a recording with my current measuring microphone the MiniDSP UMIK-1, but so far I am still in the process to include the calibration file into the recording procedure and will post it as soon as I managed.
At the same time, I had also ordered a Roland CS-10EM binaural microphone. I wanted to try out something new and although these microphones are not really suitable for neutral direct recordings of a speaker, they seem perfectly suitable to record how a speaker would sound in a particular room thanks to a rather neutral sound. You have to wear headphones to hear how the sound was meant to be, or you can try playing it back through speakers with a crosstalk cancelling filter like "Liveaudio" from the Mini Jambox.
To understand better how the listening situation was, what kind of room etc, I also did a photo of each listening position, so that you can see what I heard when I did those recordings wearing the microphones inside my ears. I am curious what you think about this kind of audio-demo, and if I should continue using it in future for further speaker comparisons too. I did not normalize the recordings, so the difference in loudness you hear is real and represents the real situation.
Again I used the JBL Charge 2 (my current portable reference despite all the distortion issues, it is one of the best sounding portable speakers which can be already had for 129) and recorded it in different rooms and positions from different distances, please have a look (and dont forget to listen through headphones):
Equally to the JBL I also tried recording some further speakers inside the shelf in the living room, I wonder if you can hear the difference in sound between all of them. I think that especially the increased stereo-separation of the EVA Blu and the Bose compared to the JBL Charge 2 and the mono Play:1 is pretty obvious:
Please let me know what you think about these recordings. Does the binaural effect work for you? Do these recordings sound more real than the previous ones I did? Which listening position does represent the most realistic situation for you?
Please leave a comment if you have further ideas!