SharpNEAT 4.1.0

Overview

SharpNEAT 4.1.0 is a minor release. The most significant change is an upgrade to the target platform from .NET 7 to .NET 8, which provides some general performance improvements.

Summary of Changes

Upgrade from .NET 7 to .NET 8; this provides some general performance improvements, particularly in the quality of the machine code produced by the JIT compiler.

Some core classes updated to make use of generic math[s] (search for uses of IBinaryFloatingPointIeee754<T>).

Upgraded Redzen dependency from 14.0 to 15.0, to make use of new generic math functions.

New Binary 20-multiplexer task. The plan is to use this to compare performance and efficacy of releases as a replacement for the binary-11 task in that role. This is becoming necessary as the speed of SharpNEAT, .NET, and CPUs in general improves, such that binary-11 is becoming too simple a task for comparing releases.

Added the 'Generative Beat Sinewave' task to the EfficacySampler app, again, to use in future release performance comparisons in place of the 'Generative Sinewave' task, which is already far too easy to provide good performance and efficacy comparisons between releases.

Fix/improvement: FuncRegressionEvaluator now handles the neural net output producing NaN and Infinity values, which can happen in cyclic networks when using certain activations, such as ReLU (because it doesn't naturally truncate the response value, thus allowing 'runaway' values.

Stochastic Efficacy Comparison

The performance (or 'efficacy') of this release was compared to the previous release (v4.0.1). The resulting best fitness histograms are shown below.

To recap, these histograms show the best fitness achieved on each of a large number of independent SharpNEAT runs, each of which terminates after one minute of execution (clock-time). Histograms are also provided comparing the evaluation counts achieved in each 60 second run, and mean genome complexity (see Efficacy Sampling for a more detailed explanation).

The PC hardware used to run these tests is listed at the bottom of the page, and is unchanged from the v4.0.1 release. However, the host PC has been upgraded from Windows 10 to Windows 11.

As in previous releases, the two problem tasks used for comparing efficacy are the Binary 11-Multiplexer and Generative Sinewave. Binary 11-Multiplexer uses acyclic neural networks, and Generative Sinewave uses cyclic networks. The plan is for future releases to move to the more difficult 'Generative Beat Sinewave', and Binary 20-Multiplexer tasks.

Results and Observations

Evaluation Count Histograms

For both problem tasks the evaluation count histograms show significant shifts to the right, with the histogram modes (peaks) shifting from around 1.1M to 1.2M for the binary11 task, and from about 4.2M to 4.7M for the sinewave task. So roughly a 10% increase overall. The histogram shapes remain similar, i.e., there no surprise changes that might occur due to accidental changes to the SharpNEAT algorithm or parameters; these histograms provide a good baseline check in that respect.

Best Fitness Histograms

The binary11 fitness histograms are very similar in shape between the two SharpNEAT versions, but with a significant shift to the right for v4.1.0, with the mode increasing from a fitness of about 1935 to >1950. Also note that the right hand tail of the v4.1.0 histogram has reached the maximum score of 2048, and indeed there were several sample that 'solved' the binary-11 task within the 60 seconds allocated.

The sinewave task histogram continues to saturate at the maximum fitness of 1000, and as such there is a plan to move to the more difficult 'Generative beat sinewave' task for comparing future releases.

Mean Complexity Histograms

The histograms remain similar between releases, with small shifts to the right for teh v4.1.0 release; this is expected for sharpneat runs that are able to perform more generations within the allotted 60 seconds.