**2015-02-10 Michael's notes (long skype meeting)**

The network is structured in 16 hyper colums, where each hyper column consists of 32 minicolums and each mini column cantains 30 cells. (16*32*30=15360 cells overall) However, we only have data of 5 cells in the first mini column of each of the 16 hyper columns (16*1*5=80 cells). We have 100.000 frames of data. (1000 Hz)

A set of 16 patterns (=4 bit) is used to stimulate the network. The first pattern stimulates the network via the first minicolum of each hyper column (using a random cell of the 5 cells), the second pattern uses the second mini columns etc.

However, given the assumed wiring of the cells ACROSS mini and hyper columns, we assume that the point of stimulation does not really matter i.e. we ignore how the network may have been stimulated.

We assume that the first stimulation takes place after 1.000 frames and that another 15 stimulations happen after this in 2.000 frames distance. (i.e. the first set of patterns has been processed after 33.000 frames.)

After frame 33.000 we assume a gap of 32.000 frames before a second phase of stimulation of the network through the 16 patterns occurs. [GE to find out what happens in the middle.]

PLAN A: For each of the 16 patterns, we try to develop a 'piece' whose character is significantly different to the other patters. If this character was seen somewhere else, one could assume a particular pattern to be at play i.e. there was a way to recognise a pattern via a particular behaviour of the network,

PLAN B: Assuming that such figures/textures could be found, we could disregard all other data and plot the development of only this one figure/texture over the complete 100.000 frames. This will likely resolve in some form of movement into organisation and out of it again.

Questions: How can we determine the characteristic differences between the responses by the network to each of the 16 patterns? What strategies of presentation can help to see those dynamic responses?

We don't have a clear timeline when the stimulation events (patterns 1-16) have occurred. It does not make sense to speculate about the cause, we can only look at the effects.

We assume there to be a difference between clusters that are formed within mini columns and clusters that are formed between cells of different mini columns.

It remains unclear if mini column clusters are a direct response to the stimulus or if they appear as the stimulus passes. However, we plan to visualise those using light gray polygons.

It is not yet clear how independent clusters can be constructed. For now, we will use David's 2-line solution, where each point is linked to the 2 points closest to it.

We hope that independent clusters will appear that sometimes are between mini column clusters while at other times, when the mini column clusters are small, they will coincide.

**2015 03 08,** Gerhard

I have reviewed the potential data (synchAlpha_100s_volts1.out) in order to relate it to the spikes data set. There is still a doubt about which is network c1 and c2. When going back to Pawel's emails, I discovered an inconsidtency, which I just emailed him about. The question is if the c1 network is the one with the lower indices or the higher ones. The former is suggested by my current understanding of the data, the latter is suggested by the naming of the variables in Pawel's Matlab file. I hope we can clarify this soon.

I could find out why we saw the uneven grouping of potential data to hypercolumns (Pawel described it is always 5 cells from 1 minicolumn in each hypercolumns). The regular grouping applies only to network c2, where there are always 5 cells per hypercolumn). In c1 we find the following grouping (number of cells in hypercolumns 1 to 16): 6, 4, 5, 5, 5, 5, 5, 6, 4, 5, 5, 6, 4, 5, 5, and 5 (cf. correlation matrix).

I created a series of overview audifications for both networks. Direct audification without time-scale manipulations speeds the data up by a factor of 44.1, i.e. 100s can be listened to in 2.26s. The clear rhythm audible are the stimulations and network responses. There are three versions per network:

(1) All cells

(2) All excited cells (lower 8 hypercolumns)

(3) All non-excited cells (higher 8 hypercolumns)

The cells are panned to the stereo panorama accoring to their index (i.e. to 80 resp. 40 positions between left and right). What can be heard cleary ist that the inputs to the cells are highly correlated because they are heard in the centre (in the centre of the head with headphones) whereas the output (spikes) are distributed over the stereo panorama. The difference between the cells of the the lower and the upper 8 hypercolums is especially pronounced with network c2.

The next two files contain the audifications of the cells per hyerpcolumn (16 times 2.26s with pauses of 0.74s, i.e. a cell group every 3s). There are between 4 and 6 cells per hypercolumn (see above) and they are also panned accoriding to they number (evenly distributed on the stereo panorama, which also allows to assess the degree of inter-cell correlation).

See also corresponding source code to the right of the sound files (from audify_potentials.scd).

**2015 02 11**, Gerhard

The PDF contains plots of all spikes of the responses to the first 16 pattners. One can see that the responses are quite different and the times between the excitations also differ a bit from one to the next excitation.

Remark: PDF to be downloaded and browsed in a viewer which allows pagewise navigation. DId not use slide show because it is too tedious to upload 16 images. Would be nice if a slide show could be generated automatically from a PDF or an archive of images.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

**2015 03 02**, David

A new version of the video of the dynamical system.

The groups of points belonging to one of the minicolums (16 in total), are used as vertices of a color filled polygon. Took quite some time to find out how to take a "random" points set and construct a polygon ot of it.

In fact, cairo (the graphics engine I am using) needs a closed path of lines in order to be able to fill the "something" with a color. First thing I tryed was to connect all points belonging to a minicolumn with all other in the same group. This didn't work as the graphics engine doesn't know which is the closed path.

So, I came up with this idea that works quite well for me now. From the positions of the points belonging to the same group (minicolumn) I compute the baricenter *b* (point with the mean *x* and *y* coordinates). Then I compute their angle with respect to the baricenter. This angle *theta *of a point* p* is equal to:

*theta* = atan2(* (py*-*by) */* (px*-*bx)* )

The resulting angles range from -Pi to +Pi. I choose the point with the smalles *theta* as the starting point for the path. The subsequent points are those with the smallest, positive difference between their angle and the angle of the preceding point. The last point is connected with the first.

Colors are chosen randomly. Each fill color has an alpha value of 0.3 (between 0 and 1.0).

In the video I've chosen not to draw the black dots representing the single neurons as in the version before. Also I don't draw the cross passing through the center.

I've experimented a bit with the parameters of the dynamical system and came up with a set which seems to produce stable and consistent results. Stable means that the system doesn't "break" and the points movement remain relatively contained into a region in the xy plane. Consistent means that for a given frame index, when restarting (resetting) the whole system the figure produced when the system comes to rest is always very similiar. For documentation, the parameters I used are:

attrition = 1000.0;

"hardness" of boundary between repulsing / attracting region = 1.4; (1.0 was at the beginning, meaning as "hard" as possible)

center "deformation" = 0.0; (off)

exponent of attracting force between points = -1.0;

Further I used a window of 500 samples (it was 1000 in the example I produced before) for computing the correlation.

In the video the starting index of the correlaton computation is printed in the upper left corner. This index was changed 20 times per second. I exported a video frame with the same rate. The resulting video has 20 fps so it is realtime. Each new frame corresponds to the next starting index for the correlation computation and setting of the new forces for the points.

Unfortunately I had to render the video very small as otherwise my hard disk is not so fast saving the new frame resulting in a asynchronicity between rendered frames and indexes.

c1 all

c2 all

c1 lower 8

c1 upper 8

c2 lower 8

c2 upper 8

c1, 16 cell groups

c2, 16 cell groups

// set paths

~root = "~/ownCloud/TP".standardizePath;

~data = ~root +/+ "data/neuro/tpbd";

~code = thisProcess.nowExecutingPath.dirname +/+ "..";

// load tools

(~code +/+ "tools/format.scd").load;

(~code +/+ "tools/dsptools.scd").load;

// network indices

~c1 = [21562, 21892, 22162, 22192, 22342, 22462, 22942, 23062, 23092,

23392, 23452, 23662, 23902, 24112, 24232, 24592, 24712, 24862, 25102,

25162, 25462, 25732, 25762, 26152, 26242, 26452, 26512, 26542, 26692,

27172, 27352, 27382, 27532, 27802, 28192, 28252, 28642, 28822, 28912,

29122, 29182, 29242, 29662, 29782, 29872, 30292, 30322, 30442, 30862,

30922, 31162, 31192, 31402, 31612, 31702, 32302, 32392, 32692, 32722,

32782, 33022, 33082, 33142, 33352, 33472, 34042, 34342, 34642, 34702,

34762, 35182, 35242, 35272, 35422, 35572, 36052, 36322, 36472, 36622,

36712 ];

~c2 = [ 2620, 2950, 3220, 3250, 3400, 3520, 4000, 4120, 4150, 4450,

4510, 4720, 4960, 5170, 5290, 5650, 5770, 5920, 6160, 6220, 6520, 6790,

6820, 7210, 7300, 7510, 7570, 7600, 7750, 8230, 8410, 8440, 8590, 8860,

9250, 9310, 9700, 9880, 9970, 10180, 10240, 10300, 10720, 10840, 10930,

11350, 11380, 11500, 11920, 11980, 12220, 12250, 12460, 12670, 12760,

13360, 13450, 13750, 13780, 13840, 14080, 14140, 14200, 14410, 14530,

15100, 15400, 15700, 15760, 15820, 16240, 16300, 16330, 16480, 16630,

17110, 17380, 17530, 17680, 17770 ];

// load potential data

~synchAlpha_100s_volts1 = ~readTPBD.(~data +/+ "synchAlpha_100s_volts1_reordered");

// put data in dictionary with cell IDs as keys

~potentials = Dictionary.new;

~synchAlpha_100s_volts1[1].do{|x|~potentials.add(x[0] -> x[1])};

// estimate mean values for both networks

~meansC1 = ~c1.collect{|i| ~potentials[i].meanF};

~meansC1.plot;

~meanC1 = ~meansC1.meanF;

~meansC2 = ~c2.collect{|i| ~potentials[i].meanF};

~meansC2.plot;

~meanC2 = ~meansC2.meanF;

// estimate normalization factors for both networks

~rangesC1 = ~c1.collect{|i| var v = ~potentials[i] - ~meanC1; [v.minItem, v.maxItem]};

~minC1 = ~rangesC1.collect(_.first).minItem;

~maxC1 = ~rangesC1.collect(_.last).maxItem;

~scaleC1 = max(~minC1.neg, ~maxC1).reciprocal;

~rangesC2 = ~c2.collect{|i| var v = ~potentials[i] - ~meanC2; [v.minItem, v.maxItem]};

~minC2 = ~rangesC2.collect(_.first).minItem;

~maxC2 = ~rangesC2.collect(_.last).maxItem;

~scaleC2 = max(~minC2.neg, ~maxC2).reciprocal;

// load potentials as buffers to server

~c1Buffers = ~c1.collect{|i| Buffer.loadCollection(s, ~potentials[i] - ~meanC1 * ~scaleC1)};

~c2Buffers = ~c2.collect{|i| Buffer.loadCollection(s, ~potentials[i] - ~meanC2 * ~scaleC2)};

// play all cells in networks, panned from left to right

{ Out.ar(0, ~c1Buffers.collect{|i, j| Pan2.ar(PlayBuf.ar(1, i, doneAction:2), j.linlin(0, 79, -1, 1))}.sum / 80) }.play;

{ Out.ar(0, ~c2Buffers.collect{|i, j| Pan2.ar(PlayBuf.ar(1, i, doneAction:2), j.linlin(0, 79, -1, 1))}.sum / 80) }.play;

// play excited and unexcited cells only

{ Out.ar(0, ~c1Buffers.keep(40).collect{|i, j| Pan2.ar(PlayBuf.ar(1, i, doneAction:2), j.linlin(0, 39, -1, 1))}.sum / 40) }.play;

{ Out.ar(0, ~c1Buffers.drop(40).collect{|i, j| Pan2.ar(PlayBuf.ar(1, i, doneAction:2), j.linlin(0, 39, -1, 1))}.sum / 40) }.play;

{ Out.ar(0, ~c2Buffers.keep(40).collect{|i, j| Pan2.ar(PlayBuf.ar(1, i, doneAction:2), j.linlin(0, 39, -1, 1))}.sum / 40) }.play;

{ Out.ar(0, ~c2Buffers.drop(40).collect{|i, j| Pan2.ar(PlayBuf.ar(1, i, doneAction:2), j.linlin(0, 39, -1, 1))}.sum / 40) }.play;

// see which cells are from one hypercolumn

~c1HyperCounts = ~c1.collect{|i| ((i - 21504) / (32 * 30)).asInteger}.histo(16); // [ 6, 4, 5, 5, 5, 5, 5, 6, 4, 5, 5, 6, 4, 5, 5, 5 ]

~c1HyperCounts = ~c2.collect{|i| ((i - 2560) / (32 * 30)).asInteger}.histo(16); // [ 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5 ]

( var c = 0; ~c1Hypers = ~c1HyperCounts.collect{|i| var r = [c, i]; c = c + i; r})

( var c = 0; ~c2Hypers = ~c2HyperCounts.collect{|i| var r = [c, i]; c = c + i; r})

// play cells belonging to one hypercolumn (4 to 6 cells, panned from left to right)

{ ~c1Hypers.do{|h|

{ Out.ar(0, h[1].collect{|i| Pan2.ar(PlayBuf.ar(1, ~c1Buffers[h[0]+i], doneAction:2), i.linlin(0, h[1], -1, 1))}.sum / h[1]) }.play;

3.wait;

}}.fork;

{ ~c2Hypers.do{|h|

{ Out.ar(0, h[1].collect{|i| Pan2.ar(PlayBuf.ar(1, ~c2Buffers[h[0]+i], doneAction:2), i.linlin(0, h[1], -1, 1))}.sum / h[1]) }.play;

3.wait;

}}.fork;