Hello HTM theory -
I've been interested recently in exploring additional mechanisms that may
enable sequence / transition learning, and in particular the possibility of
learning transitions through the existing machinery of feed forward (proximal)
spatial connectivity. There's something tempting to me about leveraging a
single mechanism for both spatial association and temporal prediction.
Based on a limited understanding of the cell level dynamics, mostly from
UTHealth's online neuroscience textbook (which seems to be a pretty incredible
resource - http://neuroscience.uth.tmc.edu/ for those who haven't seen it), it
seems like fading activations could be biologically supported in certain
contexts.
Without any further motivation than exploring what might be possible given a
spatial pooler with continuous fading activations, I've been thinking along the
lines below.
First add a few additional (not yet substantiated) assumptions:
* Fade rate can be variable per cell and influenced by connectivity, e.g.
perhaps cells with substantial distal connectivity in the same region fade
slower.
* An SDR X may be composed of a combination of core cells (X_c) which are
highly distally connected and fade slower, and peripheral cells (X_p) which
have less distal connectivity and whose activations fade faster. Some thoughts
on intuition for this at end**.
Consider arbitrary SDRs A and B. A temporal transition from A -> B might look
like the large grid in the lower right of the image below. Because the core
cells (A_c and B_c) fade slower than the peripheral cells of A and B (A_p and
B_p), A's peripheral cells' activations will have decreased more (or fully
deactivated) by the time B is active. This leads to a resultant activation in
the region with a potentially useful property: a cell in the region above could
form inhibitory proximal connections with A_p and excitatory connections with
B, and such a cell would detect only an explicit transition from A -> B.
To check this consider that (with the right thresholds) our cell can only fire
when B is fully (currently) active, A_p has faded such that the inhibitory
connections to A_p have disinhibited the cell, but A_c is still active. This
can only occur in a transition from A->B.
At this point, it seems possible that with a fairly minimal hierarchy, complex
and potentially high-order sequences (transitions of transitions) could be
learned.
<img
src="/uploads/numenta/original/1X/3d250d1638ddc1eb87aa71131bc1d11e2d6382fd.png"
width="499" height="499">
Questions on this:
* I believe I've seen mention of something like this on the forum at some point
in the past, but nothing came up with my recent searches. Is there a standard
name for this concept?
* Could spatial transition learning like this complement HTM sequence learning
/ temporal memory?
Would be curious to hear any thoughts on this, including "this isn't worth
pursuing or biologically feasible because of X, Y, Z".
Thanks!
Jeremy
_**Intuition on core vs. peripheral: One might think of core cells as composing
the central, always present features of an SDR, possibly related to the concept
of an invariant structure. Consider a tree. The verbal name 'tree' becomes
highly associated with all features that are related to trees (leaves, wood,
green), and so the SDR representing this word may contain many core / tightly
connected cells. Whereas leaves and wood, independently, have a less direct
association with each other (occur in close proximity to each other less
frequently) and all other features, and so have less distal connections._
---
[Visit
Topic](https://discourse.numenta.org/t/learning-sdr-transitions-spatially-via-fading-activations/1392/1)
or reply to this email to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discourse.numenta.org/email/unsubscribe/31d082407d6f47e1cd8142bcb1464452646c8174be280efe680a915f130f5ebe).
