2026-05-21

How good is AI hearing?

finding out if I’m to blame for being unable to follow ChatGPT’s music production advice

The complete saved Maluma study window in Ableton beside the complete native Gemini window analyzing the exported V17 MP3.

It could hear the wrong thing

I tested by giving it a Maluma YouTube song as a target. Gemini right away identified different sections and instruments.

Maluma V9: the real Ableton export still sounded like MIDI karaoke

Maluma V12: warmer, but still synthetic and rompler-like

The score was wrong

Maluma V17: Gemini 6/10, human veto, Gemini audit to 3/10

Maluma V24: the latest published step from the first-song ladder

It needed an access test

Chencho V2: rigid electronic pop, 2.5/10

Chencho V3: a clearer dembow skeleton, 3.5/10

Chencho V36: audible drums, wrong musical grammar

It could hallucinate the source

No bongos were audible.

Bachata V42: the groove becomes legible, 6.5/10

Bachata V57: Gemini inferred bongos, then admitted they were not audible

So, can it hear?

Yes. Gemini heard a surprising amount of detail and formed opinions about which instruments were present and how they were configured. The weak link was translating that perception into corrective text, turning the text into Ableton actions, or both.

The first final output was locally pleasant, though heavy on beeps and boops, but failed the actual recreation goal: it sounded nothing like the original song. The later attempts were much further off, perhaps because their genres were harder.

The judging loop made that translation problem worse. Gemini sometimes judged the wrong export or only part of a file, missed obvious problems, inferred sources that were not audible, and wrapped bad scores in plausible prose. As with a vision model, prompts had to direct its attention by time or section, audible source, rhythm, tone, balance, space and masking, then force observation before judgement. That guidance made its detailed hearing useful, but not reliable enough for unattended judging.