← Back to index

Audio Mixer: Problem when speeding up the sound

Dan Zaidan
#21319
Hello everybody!

I'm creating a new game (from scratch again, but this time on a live stream... :P ) and I have a problem with the sound mixer.
It's pretty much identical to Handmade Hero's before doing SIMD.
However when I try speeding up the sounds (by changing the sound->speed_multiplier) I get a small weird behavior that tells me the way I'm blending the samples is not 100% correct. It kind of works, but I get that noise/missed-samples artifact.
I want to be able to change the speed_multiplier every frame for some sounds.

Here is how the code is: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
    s16 *at = sound_buffer->samples;
    for (int i = 0; i < sound_buffer->samples_to_write; i++) {
        
        f32 left_sample = 0;
        f32 right_sample = 0;
        
        for (Playing_Sound *sound = playing_sounds; sound != playing_sounds + array_count(playing_sounds); sound++) {
            if (!sound->active) continue;
            
            if (sound->volume) {
                int sample = (int)sound->position;
                f32 frac = sound->position - (f32)sample;
                sample *= sound->sound->channel_count;
                int next_sample = sample + sound->sound->channel_count-1 + 1;
                if (next_sample > sound->sound->sample_count) next_sample-= sound->sound->sample_count;
                
                f32 left_sound_sample_1 = (f32)sound->sound->samples[sample];
                f32 left_sound_sample_2 = (f32)sound->sound->samples[next_sample];
                
                f32 right_sound_sample_1 = (f32)sound->sound->samples[sample      + sound->sound->channel_count-1];
                f32 right_sound_sample_2 = (f32)sound->sound->samples[next_sample + sound->sound->channel_count-1];
                
                f32 left_sound_sample = lerp(left_sound_sample_1, frac, left_sound_sample_2)*sound->volume;
                f32 right_sound_sample = lerp(right_sound_sample_1, frac, right_sound_sample_2)*sound->volume;
                
                left_sample += (left_sound_sample*clampf(0, (1.f-sound->pan), 1.f) +
                                right_sound_sample*clampf(0, (-sound->pan), 1.f));
                right_sample += (right_sound_sample*clampf(0, (1.f+sound->pan), 1.f) +
                                 left_sound_sample*clampf(0, (sound->pan), 1.f));
            }
            
            sound->position += sound->speed_multiplier;
            if (sound->position >= sound->sound->sample_count) {
                if (sound->looping) sound->position = 0.f;
                else stop_sound(sound);
            }
        }
        
        f32 min = (f32)MIN_S16;
        f32 max = (f32)MAX_S16;
        *at++ = (s16)clampf(min, left_sample*.5f, max);
        *at++ = (s16)clampf(min, right_sample*.5f, max);
    }


Do you know that could be the problem?

Thanks,
Dan Zaidan
Edited by Dan Zaidan on Reason: Initial post
Simon Anciaux
#21320 
1
if (next_sample > sound->sound->sample_count) next_sample-= sound->sound->sample_count;

Shouldn't that be different if the sound isn't looping ? I suppose you don't want to lerp with the beginning of the samples if the sound is not looping. 

1
if (sound->looping) sound->position = 0.f;

Should that be 
1
if (sound->looping) sound->position -= sound->sound->sample_count;


It's hard to help you without all the code to debug. Does the glitch only happen when you are changing the pitch or does it continue after ? Which value does speed_multiplier have ? Changing the pitch in that way will only work with small change (less than 2x, I believe it's related to the Nyquist-Shannon sampling theorem).
Dan Zaidan
#21323
Hi Simon!
Thanks for your observations. I suppose I really have to read more digital signal processing... :P

As for the code problems, I implemented your suggestions (that made it more correct), but there were 2 other big problems that ended up causing the artifact.

The first one was that the volume was only being update once per audio tick. Since it may be varying (towards a target_volume), I really should update that every sample! That improved the artifacts happening on volume change (I was testing both the volume and pitch at once... :P so the overall sound improved).

The other major thing was the first line you pointed out. Although you are probably correct and I will make it so non-looping sounds just get the last sample again for the next sample, the if (and the assignment) were just wrong. The number of samples doesn't consider the channel count, but the next_sample did! So if I have a 44100-sample sound with two channels, I must fetch all the way to sample_count*sample_count.

This is the improved code. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
s16 *at = sound_buffer->samples;
    for (int i = 0; i < sound_buffer->samples_to_write; i++) {
        
        f32 left_sample = 0;
        f32 right_sample = 0;
        
        for (Playing_Sound *sound = playing_sounds; sound != playing_sounds + array_count(playing_sounds); sound++) {
            if (!sound->active) continue;
            sound->volume = move_towards(sound->volume, sound->target_volume, sound->fading_speed/sound_buffer->samples_per_second);
            
            if (sound->volume) {
                
                
                int sample = (int)sound->position;
                f32 frac = sound->position - (f32)sample;
                sample *= sound->sound->channel_count;
                int next_sample = sample + sound->sound->channel_count-1 + 1;
                if (next_sample >= sound->sound->sample_count*sound->sound->channel_count)
                    next_sample-= sound->sound->sample_count*sound->sound->channel_count;
                
                f32 left_sound_sample_1 = (f32)sound->sound->samples[sample];
                f32 left_sound_sample_2 = (f32)sound->sound->samples[next_sample];
                
                f32 right_sound_sample_1 = (f32)sound->sound->samples[sample      + sound->sound->channel_count-1];
                f32 right_sound_sample_2 = (f32)sound->sound->samples[next_sample + sound->sound->channel_count-1];
                
                f32 left_sound_sample = lerp(left_sound_sample_1, frac, left_sound_sample_2)*sound->volume;
                f32 right_sound_sample = lerp(right_sound_sample_1, frac, right_sound_sample_2)*sound->volume;
                
                left_sample += (left_sound_sample*clampf(0, (1.f-sound->pan), 1.f) +
                                right_sound_sample*clampf(0, (-sound->pan), 1.f));
                right_sample += (right_sound_sample*clampf(0, (1.f+sound->pan), 1.f) +
                                 left_sound_sample*clampf(0, (sound->pan), 1.f));
            }
            
            sound->position += sound->speed_multiplier;
            if (sound->position >= sound->sound->sample_count) {
                if (sound->looping) sound->position -= (f32)sound->sound->sample_count;
                else stop_sound(sound);
            }
        }
        
        f32 min = (f32)MIN_S16;
        f32 max = (f32)MAX_S16;
        *at++ = (s16)clampf(min, left_sample*.5f, max);
        *at++ = (s16)clampf(min, right_sample*.5f, max);
    }


There are some things to make it better still (like changing the loops so that I iterated the samples inside the sound and not the way around to make it more cache friendly). But that'll do for now! :D


The whole code is, in fact, open sourced on itch.io! :D
https://danzaidan.itch.io/break-arcade-games-out
(And the whole development archived! https://www.youtube.com/DanZaidan )

Thanks so much for taking the time to help me out!

Cheers!
Reply to Thread
← Back to index