09:59:08 So, what I would like to do. We coordinated a bit with a lot, kind of being a part of this attempt to understand the world code and the level of single spikes in individual neurons.
09:59:24 I think we crossed parts long time, and as I tried to give you sort of historical run through, starting with about the same, about the same time that the lot started with in terms of building first pairwise models so that'll try to take it into a different direction, mainly focus on what one can use these
09:59:48 models. For what do we learn from these models beyond kind of statistical structure in the code?
09:59:55 And this has been a long path. What we try to do.
09:59:58 I study this talk, evaluating for network interactions it has been kind of an attempt that went on from the first paper and then theoretically, again, 2,010 to something that we are finishing right now in 2,022 and this work has really involved many people's sociologists the
10:00:18 acknowledgements for this later. He's on the slide by a very good Phd. Student, Mikaelic, who's now a Jamelia Newton at Ist, and Christina Zvin was the post of in our group, and later on to Syu and the data we will be using
10:00:36 is from the computer so it's kind of a theory experiment.
10:00:43 If you want contact, the first, if you were in the second being updated to theta, the pen.
10:00:55 When I go walking context like being loud for this is for the people online.
10:01:01 So I'm going to complete that now. Sure.
10:01:08 Is? Is it good? I hope it worked so. The outline of this talk goes as follows, you know I'll pose a somewhat provocative question. It was like, Ok.
10:01:20 We now know how to build this very precise data-driven models, statistical models of population code.
10:01:24 So so what right? So what do we do with them? And the idea explored here, or even the hypothesis, will be the following, and I'll make it more precise as we go along.
10:01:34 Can this inferred interactions between the neurons that we get from data?
10:01:39 Can they be also derived from some sort of an optimization or normative or ab initial principle?
10:01:47 Right, so can we derive them, and then can we confront those innovations, those predictions with the data?
10:01:53 When we do the inference. And so this is what the title of the talk is about.
10:01:56 The efficient coding for network interactions, so it's a hypothesis, and there should be a test of it, an experimental test of it.
10:02:06 And then the take one was a story of failure and fatigue.
10:02:08 On trying to get this working in the retina which I and my colleagues spend a lot of time on, and failed reasons.
10:02:15 I'll try to just summarize on one side, because nobody likes to listen about failures, and then they take 2.
10:02:22 Is this kind of revisiting the same question, but using slightly more sophisticated statistical tools and applying it to hippocampus data.
10:02:30 And I think there we can say something interesting. I hope we can say something interesting, and I hope I can convince you of that. And then in the end, you know, where do we go next?
10:02:39 Perspective slide with conclusions. Alright. So now I rewind back the time and actually goes back to the starting point of a lad's talk.
10:02:53 So this was while I was still a Phd. Student in Princeton, collaborating with Mikei Berry right in a lab with a lad.
10:03:00 Later on, and the story of that time was we use naturalistic stimuli so this was in this particular case, slightly later work was a movie of fish swimming around.
10:03:12 This is displayed to the retina, which is on the mea, and we record in this case was from 100 plus neurons.
10:03:19 To get this high-dimensional rasters that we are trying to make sense of in exactly the setting that that a lot explains right?
10:03:27 And questions were questions about population, population responses, questions about how these populations are encoding these complex stimuli.
10:03:35 And what's the statistical structure of what you're seeing here?
10:03:40 These complicated rasters. Right? So it's a series of works on this from multiple people.
10:03:45 Also, other apps, at the same time looking at mess and noise.
10:03:49 And so this is now a single slide. Rehash of what a lot was saying. One started with his realization that correlation coefficients are typically small between pairs of neurons.
10:04:02 And so, perhaps these correlations, small correlations, can be neglected, and neurons treat it as independent, and the formalism to ask this question was to build a distribution over small groups of neurons that reproduces the mean firing rate of every neuron and all the pairwise correlations which has that
10:04:20 form. I need to put it on, because I'll use it.
10:04:23 So that's a distribution over neurons. Maybe it's a slightly different notation that the last head.
10:04:28 So this is a group of neurons. And for each neuron you have one degree of freedom to fix its firing rate.
10:04:34 And you have a pairwise set of degrees of freedom.
10:04:35 I think, called the Betas, and there are jijs here, physics, and so on.
10:04:41 Traditions, so there is a pairwise interactions that you need to also tune so that you exactly matched a pairwise interactions that you need to also tune so that you exactly match in your model all the measure pairwise coe and the upshot of that was in this now
10:04:54 reproduction with plot is that even though it's now I don't go through what that means.
10:04:58 Right. So every.is a pattern, and the pairwise model really gets the observed frequencies right, whereas the independent model, which is that fails, and the upshot was right.
10:05:10 Peer, wise correlations are individually weak, but they're collective effect already in the groups of 10 neurons is strong.
10:05:17 And so you should not neglect those mode correlations. If you are trying to describe collective activity.
10:05:22 All right, so then came this sort of exercise, and the exercise of can we build bigger and bigger malls?
10:05:29 Because now we have, we go from groups of 10 runs to groups of hundreds, and so on.
10:05:34 And every one of us kind of took part in this exercise.
10:05:38 In some amount so here was at that point my attempt, which is essentially linked to what Alad was showing as a K pairwise model or see pairwise models of synchrony, so we can take larger scale recordings.
10:05:52 So this now recording for 100 neurons again. Still, the retina.
10:05:56 So you measure from the experiment. This covariance matrix of 100 by 100 neurons.
10:06:01 And the average firing rate, and you know, by some numerical effort you can from this measure.
10:06:08 Statistics you can compute the matrix of interactions of these Jij copplings between pairs of neurons, which is again 100 by 100 matrix.
10:06:18 And then these biases for each individual neuron. So this is fully specifying your maximum entropy model that exactly reproduces the data.
10:06:26 Yes, why are they? The data is, you mean in here? The data is not well, because this 0 is also blue.
10:06:38 And large in this picture. So there is a small tale of negative correlations that would be fine.
10:06:47 But no, it should be widely impossible it should be wide.
10:06:50 Yeah, more. We should be wide or should be set at white to 0.
10:06:53 So I won't go into detail just to say that when you look at this matrix that you now infer from the data you see that there is a yeah, it has a pretty complex, frustrated structure.
10:07:05 So there is quite a lot of negative interactions, not correlations and positive interactions in this matrix.
10:07:09 And we wanted to understand something about, does this have any role, or is it a bunch of statistical parameters that we should not care about?
10:07:17 You know, taken one step backwards. So when we applied this to large groups of neurons, you know, we conclude this, we includes the larger group also a lot.
10:07:31 These type of models remain powerful, they are somewhat annoying to fit numerically, because this is like Boltzmann machine learning.
10:07:36 If you want these models, start needing a bit of fixing on large populations by fixing, I mean that if you only focus on pairwise terms then when you go to 100 neurons, as we have seen, you have to start adding some higher order interactions in some way, if you want to get
10:07:54 the model that's you know, that's really matching your observed frequencies.
10:07:57 But that's fine, and this has been done in a number of ways, and there is a third thing that we haven't focused on that much.
10:08:05 I'll just put it right here that the structure of these interactions, so these are now total correlations.
10:08:11 This is noise plus the stimulus there were hints, not very precise, that the structure of these inferred interactions might be simpler if these were to account only for the knowledge.
10:08:25 Part of the correlation, not also for the stimulus part. Okay, so II will go that way. So bear with me a little bit.
10:08:33 Alright, so we could make these models for Catera.
10:08:38 And now kind of a bit more of a of a slide.
10:08:43 It's actually a slide. I keep reusing. What do we do now with these models?
10:08:45 Ok, so we feed them. They work. And so here, you know some reasons why we should look at them.
10:08:51 And the first one is a smile. If I did these mall safe we can make them better and better.
10:08:56 We can include the proper high-order terms. We can make them learn about. It's just a lot of selling them to you.
10:09:03 They fit with small amount of data. So we can do that.
10:09:07 We can even talk about generalizations, as he was showing to temporal dynamics we can use the generalization for stimulus, dependent maximum entity.
10:09:15 So that's fun. And you know that's good source of constant research work.
10:09:22 Interpretation wise. If you build a maximum model maximum entropy model, there is certain interesting things you can do with it which is harder to do with other types of models.
10:09:31 So here is a list. So you can actually bound the information that the population is encoding about the stimulus because of the maximum entropy property of the model.
10:09:42 Now we don't know whether these bounds are tight or not, but it's interesting to have a bound once you start talking about how much information in bits per second is carried by the population of 100 worlds.
10:09:55 Right. Those are not easy bounds to estimate, otherwise, because you can do it on individual neurons easily, but for a large population the majority kills.
10:10:03 If you want to do a direct estimate, you can make some directly verifiable predictions by the nature of a Max and model, let's say, pairwise pairwise mole is built only from pera, aspiration. So you can predict third order correlations, which you know.
10:10:16 You have not put in the model, and then ask how well does the model perform you can predict other interesting things, such as a distribution of synchrony, the distribution of energies, etc.
10:10:26 Etc. So these are, then all predictions of the Mall, and good validations of whether you're capturing everything or not.
10:10:33 You can try, you can construct what I call constructively disprove existing claims.
10:10:39 So there is an interesting exercise. You know. There is this idea that the neural code should be decorated, or maybe nearly decorrelated.
10:10:48 Right. So what do I mean by constructively disproving that there is a very nice illustration where you build a maximum entropy.
10:10:53 Model, which is a joint model for the whole population. And then you ask the following question, suppose I want to predict the activity of one of these neurons from the activity of the rest.
10:11:05 Once you have the joint distribution, you can do that you can predict one neuron from the rest, not knowing anything about the stimulus or anything else.
10:11:14 And if you do this, for instance, for retinal neurons from a population of 100.
10:11:18 So you predict any one of them from the ninth 9, using the mall that you fit, you are doing an amazing job of predicting its firing rate, which clearly means that these neurons cannot be decoded.
10:11:30 If you just predicted the behavior of one from the rest, ignoring a full natural stimulus that's being displayed, you don't have access to that in the model so it's a kind of a nice, so of course the code might be more correlated than if the norm's
10:11:44 in every particular. Receptive fields, but still the kind of the total amount of redundancy as the lad was also saying in this populations remains quite high, and then the 2 things maybe they also link to what Ielia might want to say afterwards you can use these models to generate new hypotheses about the structure
10:12:02 of the new code. You can explore what we call criticality in the neural code.
10:12:06 We can explore some ideas of the Bayesian code, which is that there are sort of attractors in this common space of patterns, and they encode various stimuli so these are all high high-level hypotheses that you might want to test and lastly which is what i'm going to test right?
10:12:23 Can this inferred interactions be linked to some sort of a normative theory?
10:12:27 So? Why are neurons coupled in the particular way that I inferred from data?
10:12:33 Right? So that's what we are looking at. All right.
10:12:38 And so. So I started wondering about this last question quite early, on which led to this particular paper, which was also a collaboration with the can we predict the optimal structure of network interactions?
10:12:50 And we look back at early subsystems of efficient coding, so efficient coding.
10:12:55 What it did for individual neurons is to say, let's suppose, in some sensory periphery a neuron has a receptive field, there is some structure of the stimuli coming in.
10:13:04 What should the receptive field be to maximize the information that's transmitted from the stimulus to the spike?
10:13:10 Let's say right. It's one way to phrase it.
10:13:12 And so let's do that, not for receptive fields.
10:13:15 But let's do it for interactions between the neurons.
10:13:19 So imagine now, in a toy model. So now we switch to theory or to models, not to data.
10:13:24 So I have a time on with some neurons that are interacting in some, you know, this is these are this red and green lines with some arbitrary way.
10:13:31 And I imagine that each individual neuron is driven by some type of drive.
10:13:36 Whatever this might be, some stimulus or so on.
10:13:38 High, dimensional. So there, an input, if you want that drives a bias that drives every neuron that's coming from some distribution binary G option, or whatever I have, these neurons that are responding to these drives and they can be pairwise coupled to produce
10:13:55 spiking non spiking, non-spiking responses minus 1 one here, 0 one, if you will.
10:13:59 And so the problem that we are looking to to solve is to optimize all of these N square or N choose 2.
10:14:07 Pairwise interactions here in such a way that these neurons, jointly as a population, will maximize the mutual information between their response and whatever the input is and the input is given to us it's drawn from some distribution, right the joint distribution of these drives that impact on the
10:14:26 neurons. So the model that we are dealing with superficially looks very close to the maximum model that the lad was introducing.
10:14:36 So there is this drives which now drive individual neurons.
10:14:40 There are these couplings, these are objects. We want to optimize.
10:14:43 But it's a conditional model, right? So this Jij, now, if you want, will capture the structure of noise, correlation, not the total correlation.
10:14:54 These are considered fixed for the whole population, and we want to optimise them.
10:15:00 And while the population is experiencing different drives, if you want right.
10:15:06 So that's there is a word for this class of models that we developed, which is called the stimulus dependent maximum entropy models.
10:15:16 So non-zero J. Will give rise to noise correlations in this population, and we'll somehow contribute to the total correlation the part of correlation will come from what the ages are doing, and part will come from this noise relations.
10:15:29 But it's not a noise correlation directly, right?
10:15:31 Because you go through all these nonlinear functions. If you want.
10:15:35 It's a mechanistic parameter that gives rise to no aspirations.
10:15:38 So, anyhow. So we can do this optimization problem and ask, what's the optimal structure of interactions between the neurons and no, it's always good to look at the simple case.
10:15:51 First, so the simple case is the simplest case is where you only have 2 neurons.
10:15:56 So there is only one interaction between them and then impacting these 2 neurons are 2 stimuli.
10:16:03 H. One and H. 2, and what these plots are showing in color is the optimal coupling between these 2 neurons of coupling that will increase the information transmission or maximize information transmission.
10:16:17 So going from positive to negative as a function of 2 parameters, one heap render is, how correlated are the 2 inputs that's on these axis and the other is parameter beta, which is the noise if you want or the reliability of individual neurons so high beta
10:16:35 means that the neurons are very reliable. They really dramatically by the inputs.
10:16:39 And there is not that much toasticity and small bits is vice versa.
10:16:42 There is a large amount of noise. So Beta was in the previous like temperature.
10:16:46 If your inverse temperature to one in the ising oil.
10:16:49 And so this this is not that hard to understand.
10:16:52 So if these inputs in themselves are binary, let's say 0 one there's pipes and silences.
10:16:59 Then, if the inputs are very positive, that if the inputs are positively correlated, you should always have a positive coupling.
10:17:07 But that coupling should be smaller and smaller, the more reliable the neurons are so the high beta means smaller coupling.
10:17:15 The neuron. The noise goes up. You make the coupling stronger, and vice versa.
10:17:18 If the inputs are anti-carium, and this situation changes quite a lot if the inputs are, for instance, Gaussian with their correlation, there is a regime where the neurons are very unreliable, which is this regime, and when inputs are positively correlated, so should
10:17:35 be the coupling, so the coupling should be positive.
10:17:39 So you have noisy neurons. They get correlated inputs and you should pause.
10:17:42 Ly couple them, because what you will do is just average the noise away.
10:17:47 But of course, as the neurons become more reliable, this shifts to the opposite coupling, and that is because you're shifting to the correlation limit so when the neurons becomes reliable enough, even though the inputs are correlated, you should negatively couple, them, because this will
10:18:01 now go into redundancy, reduction, the correlation limit of efficient coding that you all know.
10:18:06 Okay. Now, if you go from 2 neurons to a large population, and I have no flu why, the resolution is so bad on this doesn't happen.
10:18:18 Well, so here you see the high noise towards low noise, and this is how much information gain.
10:18:27 You will get in a population. Here is an example. Population of 10 runs.
10:18:31 If you optimize all the 10 choose to pairwise coupling, and what you see is that so?
10:18:36 This is the gaining information it's optimal, divided by.
10:18:39 If the neurons were not coupled in the low noise case, you can benefit a lot.
10:18:42 You can boost information, transmission by a factor of 2.
10:18:46 And then this game goes down to be about 1020%.
10:18:52 When neurons become renounced. So this is now the decorating limit in this type of situation, where, if the inputs are positively correlated, the optimal J, and this is really bad, actually, so the more correlation, the higher the coupling that's for high noise neurons and
10:19:11 for low mass neurons. The more correlation, the lower the inverse.
10:19:14 The coupling should be. That's d correlation.
10:19:16 That's kind of a theoretical prediction.
10:19:19 If you want in how you should connect the neurons, oftenally depending on how correlated their inputs are right.
10:19:26 So this is what it says here at low noise interactions decorate.
10:19:31 If this is possible, and you get only actually small gaining information compared to uncoupled network and a kind noise interaction at the same time as the input correlation.
10:19:43 And they might give rise to what's called the basin code, and there you can have large gains here in information transmission, right?
10:19:50 And so this is what you know, what Victorian Defian coding for network interactions.
10:19:55 And it's really trying to push efficient coding beyond optimization of feed forward receptive fields right to large networks so now the question is, how would you experimentally test that?
10:20:05 And so here is a recipe. So you record from a mural population in some system you try to extract the stimulus drives and interactions, using a stimulus dependent maximum entropy model and then you can try to compare the information that's encoded
10:20:23 by these inferred networks. Right? That half that where you remove the interaction structure, or where you optimize the interaction structure right? And try to ask, is data close to this for its close to this, or it's close to some random, connectivity that doesn't do anything for information transmission, you
10:20:40 can then also compare these relationships that's predicted between input correlation and infrared couplings.
10:20:46 And in the end, what you want actually is to repeat the same analysis on the same neurons while trying to change the signal to noise, ratio, which is beta, which is a noise parameter.
10:20:56 In your population, or by manipulating the input correlations.
10:21:01 Because we said that depending on how correlated inputs to the pair of neurons are, you should change the coupling between these neurons.
10:21:08 The really hard part in this agenda are these 2 things?
10:21:11 So this one is hard, methodologically, it's very hard to actually you know, build these stimulus, depend maximum entropy malls and control properly for all the experimental artifacts.
10:21:21 And then this, which is, how do you record the same runs?
10:21:26 What affecting their signal to noise ratio and affecting their input correlation structure.
10:21:32 Alright, so just to say that these ideas have been, now there will have been attempts to test them.
10:21:42 So one very nice attempt was actually in fly vision, looking at 2 identified a pair of identified motion, sensitive neurons.
10:21:51 And this was a a group by Axel Boors.
10:21:57 This was very nice, because this actually emerged from some discussions at Cold Spring Harbor that we all had when France Saber was a student, a wide.
10:22:04 What they did is they recorded the recording simultaneously from these neurons inferred their interactions, strength.
10:22:12 So this is he, relative coupling strength of one is what the strength inferred in the data, and then they build a model.
10:22:20 In this case it was a Glm. Mall where they could parametrically change that interaction and assess what it does to information transmission which is here so if you make the coupling stronger, the information transmission is dropping but if you make it weaker, right it also drops and
10:22:36 so there is an optimal strength of this coupling, and they compared it to the estimates directly from data, which is this, points here, and we'd remove with interference with the coupling, which is a 0 coupling this points, here.
10:22:49 And they actually managed to also do this. With 2 signal to noise trends by modulating the input contrast low and high, signal to noise.
10:22:58 And this comes out very much in line with what we have been saying.
10:23:01 There are high, you can set a non-trivial value of the coupling which actually increases information more in the Lonas.
10:23:11 Yes, which actually increases information more in the high noise scenario as we were producing still does the job in low noise.
10:23:20 But there is less of an effect, and so one could say, Okay, this is in line with with the kind of computations that we were doing.
10:23:29 And as I hinted, we try to do this for a long time by looking at simultaneously recorded gallium cells of the same type under different stimulations in the retina, and we were doing this particular analysis where we compared the responses of the population with responses where we remove
10:23:47 the noise correlation by shuffling. It's a traditional way in how we can.
10:23:50 We have repeats in the Renaissance. We can just shuffle the property to remove the no aspiration, and this did not help the mutual information so if we remove the noise correlations, the information actually slightly increased and this is reported of course in quite a lot of
10:24:05 theoretical work and literature, and so on. Just bear with me.
10:24:09 So the unsung hero of this story is Anne Vermont, who is now in Genelia.
10:24:13 We spend a ton of time trying to understand what's going on, and so on.
10:24:17 To know of it. But we'll revisit this.
10:24:20 Why we observed what we observed, and tried to interpret it later on, and later on the group of Olivia Marin Pair is with Dulis Ferrari did really a beautiful job in inferring from data the noise interactions the Jij that I was referring to
10:24:36 referring to of the stimulus dependent Mall to show that they have these beauties.
10:24:40 So these are jij between pairs of neurons.
10:24:44 But the noise just for noise, correlations, and they have this beautiful geometric structure, where close by cells are interacting.
10:24:52 And then interaction actually goes to 0 in the retina right, which has a mosaic.
10:24:55 When you go far enough. And these couplings do not depend on the stimulus type right?
10:25:01 They seem to be hardwired. Maybe they're mediated by Gap junctions.
10:25:06 In any case they are there, and the experimental work, even confirmed that if you infer them for the same neurons on one stimulus, they generalized to another stimulus which is of a completely different type.
10:25:17 For these retinal ganglion cells. So in the sense here, right there is, I mean, there is no at least dynamic optimization to the kind of stimulus they are just built into the retina.
10:25:27 Maybe they are good for natural scenes per se, as an overall ensemble, but they certainly don't change while we change the students.
10:25:35 Maybe one expected from retina too much to kind of assume that it's doing this adaptation in real time.
10:25:41 So this was our attempt number one, which didn't work.
10:25:47 Alright. So at tenth number 2. Yes, just a clarification question about the previous one.
10:25:55 When you see that the cocktails do not depend on a particular stimulus.
10:26:02 I understand that with this question. But then, what you wrote on the side of the slide, you said it's always the same for different stimulus ensembles, so I just wanted to make sure which one do you mean?
10:26:13 Or do you mean actually both that within an ensemble it remains the same for 2 different stimuli?
10:26:18 Yes, but you are using the same stimulus, or it's also the case that espres will adapt the retina to one stimulus, ensemble and fit this model and get the couplings, and then we will add the retina to a completely different example. We will fit. This model again again we will get the same set of
10:26:36 companies. So I mean, of course, the work there was limited by experiment.
10:26:41 There is a few classes of stimulated were shown all, so far as I remember, the statements they were making, so there was, I think, there was an artificial but rather complex stimulus, and a natural stimulus that was being compared so.
10:26:54 This? Is this so? I mean, they are full ensemblers in the sense that within each of these classes there was random on the retina for a long time, so that it adapted and so on. And then the Jj.
10:27:10 Could be transferred from one to the other. So our second attempt is sort of this new work, where we looked at the hippocampus data.
10:27:18 As its previously published data set which follow the paradigm of rats navigating in 2 environments.
10:27:24 The first environment was familiar to the red. So it's kind of a well, and then doesn't really matter.
10:27:32 Here, then the red one to see vanity was exposed to a new environment that it explored, and in both conditions the neurons are the same.
10:27:41 Neurons were recorded. So these are catch recordings by book of Joseph Tichori.
10:27:48 And of course, if you look at those cells that have nice place coding pro properties, this primary publication was all about you can extract what you might recognize as a place fields which are like receptive fields for the position of the animal in both of these environments in the familiar and then
10:28:09 in the novel environment. These are just a few examples. So this type of dataset was attractive to us for the following reason.
10:28:19 So first of all, we think of the physical locations of the coordinate of the animal as a as a stimulus.
10:28:25 If you want that, these particular cells are responding to, and as a population they're encoding where the animal is, I mean, they might encode many other things, but they also include a place so the response is this joint spiking activity of typically, tens.
10:28:42 To hundreds of cells that Joseph slept can do.
10:28:46 So our objective functions are what we think of being what we might think of being optimized here is the information to position, information.
10:28:56 So the information between the joint response of all these cells and the position of the animal.
10:29:02 You can also replace this with a proxy, and decod the position and ask about the quality of decodes, and so on and so forth.
10:29:08 But I'll focus on this here. And what's very interesting for us is that there are these 2 environments.
10:29:14 Ok, so there are 2 environments. There is novel and the familiar one, and one can interpret.
10:29:20 One can try to interpret, based on some previous work. Also, one can try to interpret the familiar environment as the environment where the place cells are receiving a higher quality input which we interpreted as effectively higher signal to noise, ratio on we're off from from their preceding systems
10:29:39 on where where the animal is, and from the novel environment is sort of a low signal toto-noise ratio scenario, and this I mentioned because in the theory there is this beta parameter, which is about how reliable the neurons are in encoding their
10:29:55 stimulus, and the last thing, which is very interesting for us is that when you switch to environments, many of these cells change their place field, so the placements remap.
10:30:07 What that means is that you might have had a pair of cells in one environment where the place fields were reasonably close together.
10:30:12 So their inputs would be more correlated right? Because in the same position they would both be responding.
10:30:18 And then, when the environment switches, then place fields, remap that type of correlation might change because of the remapping right.
10:30:27 And this is in our experimental test. These were the 2 things we were looking at, changing the signal to noise ratio, because theory is making predictions about that and changing the input correlation to pairs of cells.
10:30:37 Okay. So both of these things happen here. Now, you know, if we want to think about the interaction.
10:30:46 So we need a way to reliably extract these cell cell interactions, or at least have a proxy for which cells are interacting.
10:30:54 In this system. Right? And this is. And this is difficult for for multiple reasons.
10:31:01 Now yes. Sorry. This is just a very general conceptual question.
10:31:05 You are welcome to referred to the session. Is there?
10:31:10 Is a conceptual difference between what you've been doing so far and what we're doing now in that, of course, play sales do not receive location with vertical location as an implicit.
10:31:23 They did that we would already have a representation of location somewhere upstream, right?
10:31:28 In some sense you would need places. So I'm just but of course, for the rest of that, and all that.
10:31:36 That's not permission, because you do get handwritten.
10:31:37 But the retinal ganglion says that they really get an image as an input the photoreceptor's getting image is an input so I think what we do is functionally, we describe there we describe the response of a retinal gambian cell as doing directly a transformation of the input in the
10:31:53 Hippocampus. Right? You functionally describe what a cell is doing by a place field map this is not to say that they right it's not to say that those cells get X as a coordinate really mechanistically, as an input right but you can make a statistical model where you model the
10:32:08 input in that way. And your mom, okay, but okay, I mean, maybe we do model doesn't.
10:32:15 Yeah, so your model is, I guess you could interpret it as a purely descriptive model.
10:32:22 That says that my response is for some reason are dependent on this argument.
10:32:26 Too much mechanistic. Yes, I mean, this is what I can say, right? Because I'm limited.
10:32:32 But yeah, so why is extracting interactions now? Difficult here?
10:32:36 So, first of all, there is no direct stimulus repeats, as the eye is a pretty behaving animal.
10:32:43 So that's the problem. There is very strong cell cell commodulation by by various types of behavior so the animal runs fast.
10:32:52 The whole population. Structure of responses is changing. So there is.
10:32:55 You have these covariates like running velocity, that influence stuff you have synchrony, because there is, you know, the cells might be locking due to underlying rhythms in the brain, and then, you know, additionally, methodologically, it's the throat recording so there might be spike
10:33:10 sorting artifacts, or you could have errors in place.
10:33:13 Field estimation which will kind of leak into your estimates of correlation, and so on.
10:33:18 And so what we and now I mean mainly, Christina focused on is really how to properly control.
10:33:26 For this sources of correlation in order to get to the true specific pair interactions between the pair of cells that discount for all of that, for all of these factors and so the key idea was really to use my maximum entropy models in a new way to construct null
10:33:45 models that capture most of these things, and then, but they don't capture specific pair-wise interactions like Aladdin was describing.
10:33:55 So let me just show this sort of pictorially right.
10:33:57 So what do we want? So we want a model of population activity that captures the locking of each individual neuron took place, and therefore also overlaps in place fields that that controls for synchrony in the population.
10:34:12 Global synchrony of the population which might be due to the kind of a common modulatory factors like velocity or brain oscillations.
10:34:19 And so you can think of a probabilistic model that does this, from which I can draw a surrogate rusters, full surrogate thrusters, even though there is no stimulus, repeats.
10:34:30 But what that model does not contain specifically as a parameter is interactions pairwise interactions between individual pairs of cells.
10:34:37 So that's not in the model. So we don't constrain the model to reproduce individual pairwise correlations, correlations correctly as we did before.
10:34:46 Okay. But what this molecule controls for other factors does is, of course, it makes a prediction for every pair of cells.
10:34:54 What is the distribution of possible correlations right for that?
10:34:57 Like, what could I expect for a given pair of cells in terms of its correlation?
10:35:06 So it's kind of a normal prediction and then what I can do is I can take the real data, measured correlation for that pair and compare it to that distribution.
10:35:12 Using some particular. If you want p-value threshold or so on, to declare whether this pair of cells is correlated more or less compared to what you would expect, given all the other modelulatory factors.
10:35:27 So these sounds like a lot, this kind of stuff that back in the time Ken Harris was trying to.
10:35:34 Which projections? Lab. Yes, except that, you know, we used geis essentially without the.
10:35:44 Ask a question that sounds very similar to this. But that's the same here.
10:35:50 There wasn't some key differences otherwise, so can you maybe see what's the difference?
10:35:57 So I think what was very important for us, which is not written here right?
10:36:04 Was so first of all, you can frame, you can frame this whole exercise in the language of Max and Malls, which are again maximally unstructured, etc., etc.
10:36:13 But what was very important for us is to properly propagate the errors in estimation of the place.
10:36:21 Fields because we are comparing. You know, we are comparing to environments.
10:36:25 We don't have that that much data. And so you want to have a place field estimation, which is Bayesian, so that you have errors in there.
10:36:33 That then propagate to this, to this distribution, so that you can really call pairs that interact reliably or not.
10:36:43 And so this is not in here. This is Christina's work at Nips, which then goes into this.
10:36:49 But I think the question, the method is refined. The approach is very similar.
10:36:56 Credits. And so you can actually think of these maximum entropy models as very complicated shuffles that you wouldn't be able to do by hand.
10:37:04 Right. So it's a shuffle of the raster that conserves marginal properties of the cells and the synchrony, and so on, but not the correlation and the correlation is now being tested statistically right and so okay, so now i'm close to
10:37:16 the end. So what do we learn from this right? I mean, so what are the what are the options? Right?
10:37:22 So what I have here is a study excited. Or excitatory, repairs of self, and this is the number of pairs that we look at, excitatory Inhibitorian inhibitoring inhibitory. It's a complicated plot.
10:37:34 It's kind of 2 environments familiar in Zimbabwean novel is in this other orange color.
10:37:40 But the main thing that I want to draw your attention to is what is the fraction of significantly interacting pairs.
10:37:47 So this is what you can look at here, and these are fractions.
10:37:50 So for excited or excitatory, we actually find that you know, there is about 5% of cell pairs that really are outside.
10:37:59 Significantly of this new distribution. That's our model captures all the other correlation. The new model.
10:38:04 But 5% of the pairs seem to be really interactive.
10:38:06 This is higher, for ei pairs, and even higher for inhibitorory inhibitory pairs.
10:38:13 But then, when you do, a comparison between the novel and familiar environment, you can ask between novel and familiar are the same pairs interacting.
10:38:21 Or is there a change, and how strongly are they interacting?
10:38:25 You see a very different picture, which is that in this sub-network of exactly it's a lot of change.
10:38:32 So if you compare the measure of interaction in familiar versus novel, you see a pretty low correlation.
10:38:37 But if you go let's say to this guys which have quite a lot of interacting pairs you know, it's not just the same pairs, even the strength of interaction for inhibitorial, inhibitory network is actually quite stable.
10:38:47 Ok. So putting these 2 together, there is not many interacting ee pairs, but there is a large change from familiar to novel. And in contrast to here, where there is quite a lot of interacting inhibitorial inhibitor repairs. But there is not much significant check yes.
10:39:10 Alright! So we focus then, on the excitatory sub-network, just to show what I mean.
10:39:16 So these are actual place fields of my cells. And these lines here represent the strength and the sign of their corresponding interactions.
10:39:25 And so we tried to ask, how are the further interactions?
10:39:29 Are there in any way connected to the structure of the place fields, and so on. And so the key thing to skip over a lot of the key statistical signature that you can infer from data is the following.
10:39:40 So if I ask for any pair of cells, how much is their place?
10:39:44 Field overlap, which is typically how much their inputs will be correlated right?
10:39:49 If the inputs are very correlated, then these cells typically interact positively.
10:39:55 So you're have as a place with overlap grows the probability of the cells interacting with the positive interactions going up.
10:40:02 And this is more so in the novel environment that in the familiar environment and for the negative interactions, it's basically you can think of it as a random pool of weak negative interactions.
10:40:12 It's not tied to the structure of the reset.
10:40:15 Okay, or deceptive, overlap, right? So that's a so cells with overlapping receptive fields are preferentially, positively interacting.
10:40:23 Which is this thing here which now, if you think back to theoretical prediction, when would that be the case?
10:40:28 The theory was saying that if you have correlated inputs, you want the same sign interaction when we are in pretty low signal noise ratio regime.
10:40:38 So this is not the correlating limit. If this were consistent with the fission coding it's certainly not a decorrelating limit where correlated cells should be negatively coupled for the population to be correlated alright so right?
10:40:53 How do these interact interactions shape population coding? So you can build a model that has the same place called structure that you inferred.
10:41:03 And you can play with its interactions. In particular, you can consider coupling yourselves in the model precisely as in data which is a place filled overlap, and and how much the interaction should be, or you can shuffle a random interaction should be or you can shuffle
10:41:21 randomly this interactions. So there is no correlation with place field, overlap and ask what these 2 things would do for coding.
10:41:24 And so what you find is that this is now comparing this random connectivity with the connectivity that you find in the data so if you use this kind of connectivity that you find in the data, then, in the novel environment, the population can encode really significantly more bits.
10:41:43 Of information about position and in the familiar, which is the kind of high synchronized environment, the increase is still there.
10:41:50 But it's very small, and you can then also ask, What do these couplings do to the structure of the of the place?
10:41:57 Fields. So here is how a place Field would look like in this network in a novel environment, with random connectivity, and then with structural data like connectivity.
10:42:08 And what you see that these couplings are doing is they are really sharpening the marginal response properties of individual sets.
10:42:14 That's what they're doing. So you see, a sharper place fields, even though input tuning is the same of these cells.
10:42:20 The Couplings act to sharpen the marginal response properties and the result of that is, if I look at the single cell information in bits, spike, and how much information is encoded by a position.
10:42:32 Random collectivity is here structured. Data life connectivity is here.
10:42:37 And this is actually a quantity we can estimate directly from data.
10:42:39 Now from the experiment which you see matches very nicely with the structure.
10:42:44 Prediction, and then for the familiar environment, which is a higher signal.
10:42:48 Twice. Of course, all the information that are higher, because it's a familiar environment, random connectivity.
10:42:53 There is increase of information, including in the structure. And this is what also is matching the data.
10:43:01 What this says is that the structure cells and interactions, you know, compared to random interests, or even no interactions which I haven't shown.
10:43:10 They improve spatial coding, and they do so particularly at low signal to noise ratio, as we were thinking for the theory part, they are reshaped by experience, because when you change from novel to familiar environment, there is a lot of change, as I show in this excitatoryatory network, they increased this by
10:43:28 sharpening the marginal cell properties. Ok, it's not.
10:43:32 They don't increase it by smartly arranging the noise, correlation, structure they act to change the marginal properties.
10:43:40 And so, in a way, what you know, what brings these results, together with the retina, and all the kind of shuffle controls.
10:43:46 And so on. So noise, correlations in this context are a detrimental side effect, or residual, of the sharpening.
10:43:53 So if you look at the information, what happens, I put the interactions into my network I gain some information because I changed the marginal response properties of the cells, and then I lose a little bit because the noise correlations came as a necessary side effect when I put the Jij in they are not good per se and that's
10:44:11 why, if you consider a raster where you shuffle to just remove the noise, correlation, but everything else is fixed.
10:44:21 You usually see a decrease in information and I think that really makes one thing because you need to think this shuffling to remove the noise correlation.
10:44:32 There is a lot of debate in the literature about this is an operation that statistically, you can do on a raster, you as a you know, as a scientist.
10:44:40 The question is, can the circuit do that right? So the server?
10:44:43 Can it recover the noise, correlation without changing any marginal response?
10:44:47 I'm not sure, but certainly in this class of models you cannot disentangle the 2, because changing Jij is influencing also the margin of properties.
10:44:56 Yes, were also allowed to change your input changes. Yes, then, couldn't you, in principle, achieve the same?
10:45:05 Yes, you could, and then you can show that again, at least in this class of malls.
10:45:12 What that would imply is simply a trivial increase in signal to noise, ratio.
10:45:17 So, okay. But but then I guess the bigger picture question is, why do we think that networks can only tune their interactions and what they?
10:45:25 No, I think so right. So I think so. What I want to highlight.
10:45:30 I'm not saying that I'm not giving a correct answer here.
10:45:32 I'm just saying that actually, for to make this argument about what noise correlations do, or don't it's really important what the constraint is right.
10:45:40 And one way in which I think the constraint one is cheating is, if you say I can change the marginal properties in such a way that I'm just simply boosting signal noise to the network.
10:45:50 And then, of course, everything goes up in terms, information included.
10:45:56 I mean, I think one issue, though, is what is noise? Right?
10:45:59 You're assuming that what you're not understanding that network is actually noise.
10:46:05 And it's unknown to the network, and those are pretty strong assumptions.
10:46:09 So. Yes and no, I think. Yes, of course, in the brain that is correct, because we don't know what are the sources.
10:46:17 Right, I mean, is it a true stochastic source of variability?
10:46:21 Or I'm just not controlling for something, but at least in the model framework.
10:46:24 It's clear right? I mean here I do have a parameter.
10:46:25 That is playing purely impose this, like the brain doing the wrong thing is that it's certainly the case that some percentage of what we estimate as noise is not noise, right?
10:46:41 And how much and how does it influence things? Yes, so maybe to.
10:46:48 Up on the point in the particular context of a little compass.
10:46:51 One source of the parent noise, and this has been a criticism of the earth of the herbie and his work as well, because he to account this fixed recession.
10:47:06 This recession shows up as kind of Yup, yup!
10:47:10 And so my question is, and it turns out that with the geon kind of approach that Ken was using, you can, in fact, quite naturally incorporate this session. And then it turns out that a lot of to go away.
10:47:23 There was a another version of that. My question is so. In particular, he showed using giants.
10:47:30 These cross coins are there, but then it turns out that if you also allow, any, you could depend on face recession, then that explains those apparent functional functions.
10:47:41 So. My question is whether it is possible incorporate face recession in your framework, and if so, what happens?
10:47:48 So I mean, I think we can decide this inside. What we did is post-hoc.
10:47:54 Ask what would have happened if you had it. But we haven't put it in the forward modeller. Right? So so in our case, I mean, the results survive.
10:48:01 But where? What would have happened if you have it in the model from the start?
10:48:05 I don't know that alright, so I'll just wrap up.
10:48:08 This is the last slide, and then the conclusion. So so what we have now.
10:48:13 So we have a statistical signature of how interactions correlate with place field, overlap in the data we have shown that they help with information encoding about place.
10:48:25 But we haven't said anything about whether that signature, that structure of interaction, is optimal or not.
10:48:33 Right. And so you can test this by going back to the same class of model that I was just showing and instead of taking the real couplings or shuffling the real couplings, you can optimize the population to maximize the mutual information as I was showing in the theory work to really predict what what
10:48:48 those couplings should be. And so what? What do you learn?
10:48:53 So here is our you know, signal to noise ratio.
10:48:56 So this is high signal to noise, low, signal to noise, you can now optimize for different signal to noise, ratios, all the couplings, and ask how much gain in mutual information do I get and consistently with that ten-year-old theory work you see that the more noise is the more answer
10:49:14 the lower signal of noise. The more you gain in optimal networks, but you can.
10:49:19 Now, what you can now do is you can put these 2 data derived regimes on that plot.
10:49:25 So this is the regime that corresponds to the familiar environment in the data this is the regime that corresponds to novel environment.
10:49:32 You see those increases, they are not as high. Normally as there are in our optimized networks.
10:49:38 But of course it's very hard for us. The exact quantity of comparison, because we are not recording from the full people campus population right?
10:49:46 And we're using a smaller storage population here.
10:49:49 But I think this is the really interesting result. When you do the optimization, and you ask in the pure ab initial optimization setup, what should the couplings be as a function of place filled overlap, you get this type of curse, so maybe higher overlap means you need to have higher coupling and
10:50:12 there is a tail that becomes a bit negative, and this is for a high signal to noise regime, and you can directly compare that it has nothing to do with data.
10:50:20 So this is computed to the structure that I was showing you, which is inferred from the data.
10:50:27 So this is now purely inferred. Right? So what is the dependence of the coupling on the tuning similarity?
10:50:33 And even though this axis is not exactly this axis, because we have no way of bringing the model and the data into that kind of quantitative agreement that the structure, you know qualitatively, is very very similar.
10:50:48 I mean, you could even make it quantitatively similar by one rescaling coefficient. Right?
10:50:52 And so I think that this key signature, this key structure in the network is really consistent in the data in the Ab initio optimality prediction, and actually something that I'm not showing is that if you construct synthetic networks of neurons and couple them by simply taking this as a
10:51:11 connectivity rule. I say, I don't optimize anything.
10:51:13 I just whenever there is a high overlap of receptive fields, I'll throw down a strong coupling between those 2 neurons and weak coupling when there is nothing and I won't even optimize these couplings.
10:51:26 I'll just do that as a simple proxy.
10:51:28 You actually find that you recover most of this information gain just from getting this topology of who you should couple, and not even fine-tuning the couple in strength. Ok.
10:51:40 So you can get quite far by just relying on this without any extra optimization.
10:51:45 Alright. So, to conclude, you know, methodologically, I think what we really it really took us a long time to do this essential improvements in data analysis to really reliably extract specific cell selling drugs and enable this sort of side by side comparison of model and data it's the same
10:52:01 model class that we are working in, both in more land and data.
10:52:05 It's both stimulus dependent, Max, and small, and then you can kind of compare them side by side.
10:52:10 So what I think this shows is that network interactions in the excitatory subnetwork adapt to changing noise, which is the familiar version novel. So it's changing signal to noise.
10:52:20 And they adapt to changing, of the input correlations, because place fields are remapping and the ee network changes a lot between the 2 conditions.
10:52:29 This effect really optimizes spatial information. And so I think it's a a putative example of this predicted network efficiency, cooling.
10:52:42 Mind you, we are not in the decorating regime of the efficient coding right.
10:52:46 So this is what you do not predict in the sensory periphery I find that interesting.
10:52:51 I find it kind of resonating with some of the work we have done with Jonathan before applying efficient coding to texture, perception.
10:52:58 We also, you know, the predictions work extremely well, but not in the regime where you should be decorating, but within the regime, where input noise is high.
10:53:06 And so that is maybe an interesting regime, because in the sensory periphery, the only time I know it was this regime is applied is to predict what happens to receptive fields at very low light right where they should lose their surround and usually just average but maybe for the central regions this other
10:53:25 regime might be more relevant. And so then this is sort of this is more of a caveat.
10:53:30 Reflecting the conversation with Matthew. This is just to say, you know, do noise correlations, help, or hurt, on which there is a ton of literature also in theory, right?
10:53:41 I mean, this is a question to be asked you know very how to say carefully right, because it really depends on what your fixing and what you're not fixing and what you're allowing yourself to do, and whether you're taking mechanistic constraints of some sort in your network into account
10:53:53 right, which ultimately, I guess, one should.
10:53:58 Alright. I think I'm done so this is what I'm interested in.
10:54:02 Are there, you know, further potential kind of is there further potential for normative theories that are based on coding and on information maybe a of efficient coding flavor.
10:54:15 But they go beyond sort of sensory periphery, right?
10:54:17 Maybe as this, as an example. Alright. So let me finish.
10:54:22 Here and there is any questions. Thank you.
10:54:34 Maybe.
10:54:39 The familiar versus an adult environment.
10:54:45 The edible standard. What's that? The media environment? Because it has a different distribution of of place?
10:54:52 How long does it take to get to that? Let her off of that distribution?
10:54:58 So if they were to spend more time with the new one.
10:55:02 How long? Yeah, so this is. So this is something we wanted to do, but couldn't.
10:55:06 So what we wanted, of course, is to be able to say something about the time course of what happened, of the changes and what we are limited.
10:55:17 I mean, even though Christina came up with these very good estimates of place fields that give you very reliable estimates.
10:55:23 10 min or so. If you really want something nice. We didn't dare, and I don't think we could have gotten a temporal profile of what happens with these interactions to get a timescale that would be very interesting.
10:55:36 But it seems like there were some hints which are not in the paper.
10:55:40 That placeful remapping happens really quickly. And then there is another lower timescale on which this interions are how to say adjusting to the romplace fields. But you know, we didn't kind of dare make actual claims on what these times guys are yeah.
10:56:00 Alright, but right. So you read, when do we reconvene?
10:56:09 You are just somewhere else.
10:56:16 So then, so that means that the presentation and discretion will be consistent.