Activation functions vary by application, but most assume normalized inputs and outputs to the range of -1.0 and 1.0 (Bipolar Sigmoid, Hyperbolic Tangent) or between 0.0 and 1.0 (Gaussian). Other activation functions below do not assume normalized input (Sin, Cos, Tan, Step, Ramp, Spike, Inverse) and thus are useful for applications in which inputs are not restricted to the -1.0 to 1.0 range.
So why use CPPNs when theoretically, an ANN composed of only Sigmoids or Gaussians can potentially approximate any function? Because with multiple activation functions an ANN obtains variation more quickly and may possibly evolve toward a target sooner. CPPNs are useful for Interactive Evolutionary Computation (IEC) and Evolutionary Art applications where obtaining variety quickly and minimizing evolution time for users is a priority. Additionally, some of the activation functions presented are linear, enabling Evolutionary Art programs to easily generate straight lines, which is problematic for ANNs composed of strictly non-linear functions.
Note that in the code for sin(x), cos(x), tan(x), and tanh(x) you must #include "math.h" for C language or #include
Code: The discontinuity of tangent produces interesting results in NEAT Particles, therefore it may be useful in other Evolutionary Art applications. Since the standard C/C++ tan() function can return NAN (not a number), +INFINITY, -INFINITY, or a number, you must handle each case. I arbitrarily chose 10000.0f as a large number to return.
float tangent(float x)
x = tan(x);
if(x<10000.0f && x>-10000.0f) return x;
if(x > 10000.0f) return 10000.0f;
if(x < -10000.0f) return -10000.0f;
Equation: sinh(x)/cosh(x) =
float bipolarSigmoid(float x)
return (1.0f-exp(-x)) / (1.0f+exp(-x));
Code: You can greatly simplify the Gaussian formula as below if you don't mind that it approaches 1.0 at zero unlike the classic Gaussian function.
float gaussian(float x)
return exp(-1.0f * (x*x));
Equation: 1 - 2*(x-floor(x))
float ramp(float x)
return 1.0f - 2.0f * (x-floor(x));
Equation: if(floor(x) is even) 1.0 else -1.0
float step(float x)
if((int)floor(x)%2 == 0.0f) return 1.0f;
else return -1.0f;
Equation: if(floor(x) is even) 1 - 2*(x-floor(x)) else -1 + 2*(x-floor(x))
Code: Basically a pointy version of sin or cos.
float spike(float x)
if((int)floor(x)%2 == 0) return 1.0f - 2.0f * (x-floor(x));
else return -1.0f + 2.0f * (x-floor(x));
Code: Since inverse flips the function in the preceeding node it may be useful for producing mirror images or tranforms.
float inverse(float x)