One of the basic problems you might want to solve in doing PCM synthesis is limiting a signal to the range between -1 and +1. Obviously you can just use a mid() function to get hard clipping - the value just locks to the endpoints - but this often has unpleasant side effects. For example, if you hard clip the output, the sound will snap abruptly from smooth to harsh as the volume increases, with very little transition between these. So, people often want to round off that transition, let the sound get gradually harsher, and one of the smoothest options is to use the tanh - hyperbolic tangent - function. Looking at a graph of this function, you can see that it smoothly transitions from reproducing the original x value to squishing it between -1 and 1, just as we often want.

So, I did a bit of experimenting today with sine waves and came up with the following code snippet:

 ```--to use table: -- tanh = lut[x\0x.01] or sgn(x) function lut_tanh() --lookup table of tanh -- between -3 and +3 -- with step size 1/256 local tanh={} for i_x=-768,768 do --naive formula is --tanh = 1 - 2/(1+exp(2x)) local x=i_x>>8 --=i_x*0x.01 --inspired by https://www.math.utah.edu/~beebe/software/ieee/tanh.pdf formula: --1. from 0 to x_small=0x0.01bb, tanh(x) = x --2. for x>=0x5.3300, naive formula returns 1 if abs(x)<=0x.01bb then tanh[i_x]=x else tanh[i_x]=1-2/(1+0x2.b7e1^(2*x)) end end return tanh end```

To my ears, with these values, driving a sine wave with an input volume as low as 0.03 sounds only about as bad as the original sine wave does with so few bits to work with, and driving all the way up to 16 there's not a lot of artifacting to the distortion. With some antialiasing you could probably do even better than that.

This function uses 1537 entries in the lookup table, which I think is about 1.4% of Lua memory. I know my ears aren't very trained when it comes to analyzing synth sounds, so feel free to edit this if it seems like too much or not enough, but it seems to me like a fairly practical compromise for PCM slingers.

P#140078 2024-01-13 19:25