Sigmoids Distinguish More Efficiently Than Heavisides

Every dichotomy on a 2k-point set in ℝN can be implemented by a neural net with a single hidden layer containing k sigmoidal neurons. If the neurons were of a hardlimiter (Heaviside) type, 2k 1 would be in general needed.