Cellular Automata with FeedForward Neural Nets as Neurons (and chemicals)
hey
Have you guys heard about fruit flies lately?
Yeah they apparently managed to map out an entire fruit fly’s brain neuron structure.
Here’s the link to the research if you’re interested.
This is a pretty good milestone in of itself, and what it says is basically this.
…
Our AI connectomics tools include flood-filling networks, which use convolutional neural networks to start at a single pixel and identify all other pixels that are part of the same object. In 2019, our Connectomics team released an initial, fully-automated reconstruction of a female fruit fly brain. By 2020, our team and collaborators released a human-verified map of half a female fruit fly brain with 25,000 neurons and 21 million connections, a record at the time. Meanwhile, the team was already working on the full, verified brain map for a male fruit fly, which is now complete. …
This level of progress is absolutely staggering and what interests me even more is that they managed to produce a DIGITAL representation of the fruit fly brain for anyone to use, and IT ACTUALLY produces the same behaviour as that of a fruit fly. How is this possible?
Well the brain of fruit fly technically doesn’t hold any “memory” of it’s past life, that’s bull. So the thing is that the neurons are wired in such a way that every input that travels through the synapses, gets gated such that it results to the same behaviour as when the fruit fly was alive. Think of thousands of synapses managing thousands of “if this then do that” statements, but invisibly.
what my thoughts on this were
My personal thoughts were kind of mixed. Well you see, the researchers made the digitally mapped brain public for anyone to access.
Bunch of people brought it into minecraft, some guy even hooked up earphones into the digital fruit fly’s ears to music
This got me thinking, that if thousands of years of evolution could eventually land at the same solution as complex gating mechanisms, what are the chances that we could try to simulate that as well, and create better brains in the process.
The main problems with most neural networks today is this -
- Every neuron speaks to each other. There’s no localization, and every single input takes a HUGE load to process
- Neurons are extremely complex in nature. They don’t only rely on synapses, they rely on chemicals and release chemicals in return
- There’s multiple types of neurons and fibres that introduce complexity and used to speed up things
- The spikes are actually delayed in nature, each synapse has a strength that can decide how fast the spike goes, and how strong the spike will arrive at the other neuron
The actual thing is that some really smart people in history sat down to do “biomimicking”. They tried to emulate the concept of a brain through calculus and non linear polynomials. That’s pretty good for what it served.
And for a long time, we actually did manage to do good with it. The architecture was just complex enough to trigger complex behaviour and at the same time not too complex to make. It was purely based on the number of neurons you give the model and how you treat them while training.
so… what’s your point?
Something in me instinctively kind of sought to get a better answer. See the thing is ML models cant replicate complex behaviour without a lot of tuning and training. And that’s not their fault.
It’s the fault of the fundamental building blocks. Evolution in nature works with an infinite space with varieties of compounds and chemicals to make things that aren’t conceivable to us at the time, and then fit into the creature what works and rewards it.
Maybe we need some of that complexity, maybe its exactly that what makes it easier to fit more complex behaviour into a network. That’s probably - and this is just my hypothesis - how we can cut down on compute.
Infact the entire point should be not to have more gates, it should be to have lesser but SMARTER gates.
So i connected a few ideas
John Conway’s game of life
John Horton Conway (26 December 1937 – 11 April 2020) was an English mathematician. He was active in the theory of finite groups, knot theory, number theory, combinatorial game theory and coding theory. He also made contributions to many branches of recreational mathematics, most notably the invention of the cellular automaton called the Game of Life.
This guy is a living legend. Everyone in the scientific field knows him for the Game of Life.
So if you’re not aware, let me tell you what the “game of life” is.
Take a nxn grid of squares. Place some living cells in it.
Every generation you do all of these things -
- Underpopulation: Any live cell with fewer than 2 live neighbors dies.
- Survival: Any live cell with 2 or 3 live neighbors lives on to the next generation.
- Overpopulation: Any live cell with more than 3 live neighbors dies.
- Reproduction (Birth): Any dead cell with exactly 3 live neighbors becomes a live cell.
Now for a really small amount of living cells - this doesn’t do jackshit. Infact, you could spend the whole day staring at a blank screen and see nothing.
With a lot of cells almost nothing interesting happens, it looks messy, it looks chaotic and it looks dumb.
But if you let it run for a while and at a grander scale…
It can make something absolutely beautiful
It’s absolutely bonkers what some amount of emergent behaviour can come from such a simple ruleset. Thus Conway’s Game of life actually stands as one of the most inspirational and sophisticated ways of saying this
Yeah, sometimes chaos causes order
This is the principle upon which evolution happens. How something out of nothing forms.
Now obviously this is a much more extreme example of order and mostly things that are formed are usually not that ordered, they’re small bits of order.
Anyways
Neural Evolution of Augmented Topologies (NEAT)
NeuroEvolution of Augmenting Topologies (NEAT) is a genetic algorithm (GA) for generating evolving artificial neural networks (a neuroevolution technique) developed by Kenneth Stanley and Risto Miikkulainen in 2002 while at The University of Texas at Austin. It alters both the weighting parameters and structures of networks, attempting to find a balance between the fitness of evolved solutions and their diversity. It is based on applying three key techniques: tracking genes with history markers to allow crossover among topologies, applying speciation (the evolution of species) to preserve innovations, and developing topologies incrementally from simple initial structures (“complexifying”).
If you didn’t understand any of that, it’s alright
All it says is that we borrowed evolution and put it on our computer
You make a bunch of creatures. You give them a simple task like, say, walking
At first they all suck at walking. They’ve been given legs they can’t control, and feet that are weird to look at.
But some of those creatures manage to walk a step or two. We love these creatures!
This generation finishes, and in the next generation, the creatures that performed the best are allowed to reproduce.
Those reproduced creatures are having a small catch - mutation!
Now repeating this process in the next generation - the test decides whether this mutation helped the creature, i.e “managed to walk further out” or it just killed the poor creature “gave it genetic heart disease”
Believe it or not this works!
- What this excels at - when a fitness function is known i.e we know how fit (or good) the creature is at all times.
- What this absolutely hates - a complex task whose reward is extremely hard to get.
This is a method OTHER than backpropagation that gives similar if not better results when you’re training creatures sometimes.
So you may ask where i’m going with all this?
To create a simulation environment
The neuron
The major issue at hand
What is a neuron? How complex is it really?
You see the normal definition of a neuron in neural networks is generally W1 X + B1 (ok thats a synapse technically) but in real life neurons are more complex than they let on, they have a bunch of things they come in accordance with -
- They react according to the chemicals around them
- They produce chemical volume transmissions
- Some neurons actively fire more than others or less than others based on certain properties
- Some neurons fire WHEN there are certain chemical inputs as well
- Some neurons DON’T fire when there are chemical inputs
These complex gating mechanisms are actually crazy in nature. And they were perfected over millions of years of evolution. And it is these mechanisms exactly that help run a brain at only 20 Watts and do millions of little reactions per second.
You know what else is a complex gating mechanism? A feedforward neural network. This is why all neuron inputs are going to be fed to a small mini “ffd” and outputs will spit chemicals and trigger firing.
Yeah a non linear polynomial does basically fit our use case.
1
2
3
4
5
6
7
8
9
# FFN input: [potential, chem_1..chem_n]
# FFN output: [fire_decision, release_1..release_m]
self.n_in = 1 + number_of_input_chemicals
self.n_out = 1 + number_of_output_chemicals
self.W1 = self.rng.standard_normal((self.n_in, hidden_size))
self.B1 = self.rng.standard_normal(hidden_size)
self.W2 = self.rng.standard_normal((hidden_size, self.n_out))
self.B2 = self.rng.standard_normal(self.n_out)
1
2
3
4
def _forward(self, x):
"""run the FFN on an input vector, return the full output vector."""
a1 = self.sigmoid(x @ self.W1 + self.B1)
return self.sigmoid(a1 @ self.W2 + self.B2) # sigmoid is probably one of the best introducers of non linearity, might leakyRELU in the future but this is worth it for testing
The synapse
A synapse has features which are pretty simple
- They’re basically decided by which neurons fire together, they communicate via chemicals to “reach out” and grab each other
- They have a delay in how much time a spike reaches another neuron
- They have a strength
turns out if you keep a buffer, you can achieve the delays.
1
2
3
4
5
6
7
8
9
def send(self, fired: bool):
"""Sender's decision this tick: schedule it to land `delay` ticks from now."""
self.buffer[-1] = fired
def advance(self):
"""Move the buffer forward one tick; whatever reaches the front is the arriving spike."""
self.spike = bool(self.buffer[0])
self.buffer = np.roll(self.buffer, -1)
self.buffer[-1] = False
Chemicals
wait but the direction this is going towards is simulating compositional chemistry from scratch?
Here’s a good idea - we DON’T need to simulate proper chemicals like dopamine and serotonin, we just need “placeholders” that neuron identifies as different chemicals.
Since neurons are essentially feedforward networks, all we need is to add extra inputs to the design - this separates chemicals as just different enough to trigger the complex gating we need from them.
Another thing, i’ll be using exponential to try to map how the chemical propagates outward in a ring, sort of a “radius of proximity” like in real life, initially it was supposed to be supported by 1/r^2 but well it’s 2d, not 3d so i decided against it for now.
in the neuron class
1
2
3
4
5
6
...
self.fired = False
self.chem_inputs = np.zeros(number_of_input_chemicals)
self.chem_release = np.zeros(number_of_output_chemicals)
self.incoming = []
...
in the input pass
1
2
3
4
...
x = np.concatenate(([self.potential], self.chem_inputs))
a2 = self._forward(x)
...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
def chem_tick(self, neuron: Neuron):
# some chemistry shit going on here
neuron.chem_inputs *= self.chem_decay
#print(neuron.chem_inputs)
for n2 in self.neurons: # ideally this should run on maybe 10 neurons, as we have smarter neurons, ill switch to grid quantization and diffusion cells later in a newer update but for proof of concept this is good
if not n2.chem_release.any(): continue
if id(neuron) != id(n2):
distance = math.hypot(neuron.pos[0]-n2.pos[0],neuron.pos[1]-n2.pos[1])
#print("chem inputs: ",n2.chem_release,", self ticks",self.ticks)
neuron.chem_inputs += n2.chem_release * np.exp(-(distance**1) / self.chem_range) * self.chem_amp
neuron.chem_inputs = np.clip(neuron.chem_inputs,0,5)
The setup
Yeah that went about as good as one could hope
The idea is that if you hook up these hidden neurons to a bunch of input neurons, they execute some actions and then signal to output neurons
In real life they’re called motor neurons - like your arms legs, whatever you want from a half lobotomized creature on a silicon chip running bloated software.
Another thing you might’ve realised if the entire network is constantly sending signals it immediately becomes a noisy mess, much like the randomized conway’s game of life
- In actual neurons this is fixed using something called “Homeostasis”, which in broad terms is something like -
Hey if we’re firing too much, lets calm down for a bit and save resources.
in the processing phase -
1
2
3
...
self.threshold += self.homeostasis_tau * (self.rate_estimate - self.target_rate)
...
The general case is to show that a signal can travel from input neurons to output neurons.
What now?
Well this is impressive but it’s absolutely worthless if we have to design these brains ourselves. So you remember how we ranted about NEAT algorithm for a bit? We’ll now be creating creatures - each with a different brain.
There will actually be 2 NEATs - each one decides a specific brain structure and isolates special properties -
The outerNEAT
- This specific NEAT will reward good neuron placement - sort of like building localized centres for the brain
- Why? Because you don’t want to create localized brain centres by inducing a cost to creating connections - it should be a natural choice not a forced one, this encourages NEAT to encourage and find the correct answer no matter what. Which is pretty lenient.
The innerNEAT (Fine tuner)
- So when the creature has a brain - it goes through mutated “tuning”
- We try mutating the synapse placement - sort of like how real brains learn, by killing old connections and making new connections.
Shaurya, why does this need 2 separate NEATs?
Well you see, the combination of both these NEATs is testing something we’ve actually never thought of before.
The NEAT deciding neuron placement basically spots if neurons placed in proximity sharing chemicals in volume could be an optimal design for the creature, relations that involve multiple neurons at once
The fine tuning NEAT takes advantage of these proximities to push the brain to it’s tuned version, neurons that only talk to SPECIFIC neurons
These 2 building blocks ensure extremely complex gating mechanisms, they could maybe recreate XOR, AND and OR bitwise operators for now maybe.
Another thing before we go any further, we will define the TYPES of neurons we will give giving outerNEAT to build with. We’ll create a standard “palette” of neurons that NEAT has to paint from, which have randomized parameters and different properties.
1
2
3
4
5
6
7
8
9
neuron_palette = [make_sane_neuron(pos=(0,0),
color=(random.randint(0,255),random.randint(0,255),random.randint(0,255)),
rng=rng,
threshold_margin=0.15,
homeostasis_tau=float(rng.uniform(0.5, 0.9)),
potential_leak=float(rng.uniform(0.1,0.3)),
spike_leak=float(rng.uniform(0.1,0.3)),
input_gain=float(rng.uniform(0.5,3)) ) for _ in range(NEURON_PALETTE)]
The innerNEAT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# do some tuning in synapses.. to like 20% of the connections (adjustable)
final_creature = self.original_creature.clone()
for synapse in final_creature.all_synapses:
if random.random() < self.randomize_synapse_weights: # 20% chance of changing up the weights
raw_weight = float(random.uniform(0.4, 1.2))
synapse.weight = -raw_weight if synapse.sender.is_inhibitory else raw_weight
# randomize wiring
for a in final_creature.neurons: # right now only 70%
if random.random() < self.randomize_neuron_wiring:
if len(a.outgoing_synapses) >= 1 and random.random() < 0.5:
a.delete_synapse(random.choice(a.outgoing_synapses).receiver)
else:
b = random.choice(final_creature.neurons)
raw_weight = float(random.uniform(0.4, 1.2)) # magnitude only, always positive draw
weight = -raw_weight if a.is_inhibitory else raw_weight
if not ((a.is_input_neuron and b.is_output_neuron) or (a.is_output_neuron and b.is_input_neuron) or (a.is_input_neuron and b.is_input_neuron) or (a.is_output_neuron and b.is_output_neuron)):
a.add_synapse(
b,
weight,
delay=self.delay_from_distance(a, b),
)
# add
self.finetunes.append(final_creature)
So, i tested the fine tuner with one simple task - given randomly placed neurons, make it so that when an input neuron fires, an output neuron gets fired
This looks like a pretty small achievement, that’s because it is, I just needed a way to know that my algorithm worked properly
So then i made a similar task - Each corresponding input should be wired to the corresponding output neuron
Then it surprised me - it managed to pull it off
The only rule was that no input neuron could connect to any output neuron directly - that’s cheating and i don’t support it. Each fine tuned creature was given extra points for whenever it completed a connection so it was able to walk up the staircase of difficulty with ease.
Speaking of the staircase of difficulty.
You know how you go to school for 12 years, then do a degree in computer science for 4 years, only end up jobless and miserable?
Well, NEAT needs a curriculum as well -
- The core idea of NEAT is stumbling into better creature designs
- when your search space is too big and your idea too complex, it can take too much time and sometimes be impossible to hit that design.
- That’s why you level UP the creature slowly - so instead of dropping literary analysis of communist manifesto on the poor creature, you first start with kindergarten storybooks.
The chemicals
Before we go forward any further, we haven’t actually added any chemicals yet. In a real brain - chemicals decay over time, they get cleaned up immediately by your brain so they don’t lie stagnant.
These chemicals affect the plasticity of neurons and their ability to form connections most of the time.
We’ll be taking the shorter route - we won’t be affecting the ability to form connections.
The outerNEAT
Alright so buckle up because here’s where it gets interesting -
- Technically the outerNEAT isn’t really a creature in of itself, it’s more like a blueprint of how the creatures neurons are placed, we call this a “skeleton”
- If you look closely you’ll realise that the reward of the outerNEAT is directly the result of the best creature’s fitness formed by innerNEAT.
- innerNEAT is given a small amount of “passes” or time to fine tune itself to the best version possible
- this creates an inherent pressure to get to the best answer possible as fast as possible
- This pressure may make outerNEAT utilize chemicals over synapses sometimes
So after 10 skeletons per 5 outerNEAT generations combined with 5 finetune generations, i finally saw the NEAT use the chemicals as a choice to build the brain
It’s actually wild because the 2nd input neuron fires both the 1st and 2nd output neurons, and i don’t know somehow my evaluation script broke and it wasn’t able to stop it from doing that, it triggered not the “crystal clear” conditions as it had in automated testing.
But the other 2 neurons were fine, and they utilized the chemical signals well.
What does test mean?
This structured learning can use chemicals as a fundamental block for brain design - albeit it’s shabby, but at a bigger scale it should be able to pull off much cleaner ways
Also means that synapses can WORK in tandem with chemicals instead of either replacing each other
The network CAN do complex gating mechanisms - i checked via the playground simulator i made for the neurons - they’re able to sync well with the chemicals, and i wonder what kind of gating mechanisms they’ll form when i introduce more than just one chemical.
TODO (rough)
- will add it so that instead of from scratch random creatures, i mutate 20% of the neurons to be positionally different or of different types
- should cut back on the general noise allowing for more refined structures
- mutation parameters that adapts to the training phases would be interesting
- need to add a more complex task to solve, more neurons in a larger space
- or more possibly just more of the same but longer generations (more than 5 previously)
try to recreate functions like xor, and etc.
- sooner than later will implement an active AGENT focused task that will involve a physical space around which the model can move and get rewarded for
- need to add a cuda kernel that will speed up NEAT, ill migrate to a faster language sooner or later when the proof of concept works in its entirety and is worth testing
- eliminate redundant parts of the scripts that take up processing power
On scaling
As you can guess the amount of compute it takes for chemical transmissions and neurons can get heavy (hard to believe i know), and to every extent its very possible to push the hardware to it’s limits, what i’ve built is a prototype and nothing else, the actual design can be optimized A LOT more further, here’s key notes anyway
- I think its important to remember that the problems an optimal brain design solves is cutting down compute and finding efficient patterns where needed like for eg. drone navigations, this is why we placed outerNEAT
- Secondly, the underlying insanely random nature of NEAT sometimes tend to find solutions that were previously not reachable by any other methods and dropping compute on having NEAT find an optimal brain instead of pushing a brain towards perfection with gradient descent is what a good use case looks like.
- To elaborate more on second point, i think the goal should be on the lines of Capex vs Opex - which is essentially high setup cost, tiny running cost - the cost of using advanced randomized gating systems to let evolution decide how it can be optimized.
- Thirdly, the level of complexity of the fundamental blocks we’ve provided NEAT makes it so that with lesser neurons it’s able to achieve more, which is basically narrowing down the search space - smarter neurons for the outerNEAT provide searchable spaces where throwing huge amount of neurons was initially the approach.
- Essentially the complexity of the environment bridges the gap between complex tasks and NEAT
Conclusion
Alright so i’ve been working on this for a while now and i want to take a break and come back to it later with some fresh ideas.
Maybe this could be one of those slow build worth the wait projects.
Anyways bye.