Mattar, A. A. G., Ostry, D. J.
Studies on generalization show the nature of how learning is encoded in the brain. Previous studies have shown rather limited generalization of dynamics learning across changes in movement direction, a finding that is consistent with the idea that learning is primarily local. In contrast, studies show a broader pattern of generalization across changes in movement amplitude, suggesting a more general form of learning. To understand this difference, we performed an experiment in which subjects held a robotic manipulandum and made movements to targets along the body midline. Subjects were trained in a velocity-dependent force field while moving to a 15 cm target. After training, subjects were tested for generalization using movements to a 30 cm target. We used force channels in conjunction with movements to the 30 cm target to assess the extent of generalization. Force channels restricted lateral movements and allowed us to measure force production during generalization. We compared actual lateral forces to the forces expected if dynamics learning generalized fully. We found that, during the test for generalization, subjects produced reliably less force than expected. Force production was appropriate for the portion of the transfer movement in which velocities corresponded to those experienced with the 15 cm target. Subjects failed to produce the expected forces when velocities exceeded those experienced in the training task. This suggests that dynamics learning generalizes little beyond the range of one's experience. Consistent with this result, subjects who trained on the 30 cm target showed full generalization to the 15 cm target. We performed two additional experiments that show that interleaved trials to the 30 cm target during training on the 15 cm target can resolve the difference between the current results and those reported previously.
Disclaimer: I was a reviewer of that paper that rejected it but my criticisms were only partially answered… Here they are
In this paper, the authors address the question of generalization of dynamics learning across changes in amplitude. They report that subjects generalize from larger to smaller amplitudes but not vice-versa. They hypothesize that this asymmetry arises because increasing amplitude forced the subjects to use higher velocity than experienced during the training, and they could therefore not use what they had learned at lower velocities. They also performed a couple of control experiments to reconcile their data with the data from Goodbody and Wolpert who addressed the same issue ten years ago.
Unfortunately, the conclusions of the paper are based on a flawed statistical argument (comment #1) and most of the analyses are poor (wrong parameter, no normalization for speed differences, …. see comments #2). Therefore, the data might tell something else than what the authors concluded.
Major comments:
1. The statistics are often erroneously interpreted and there is a lack of direct comparison between the groups. One logic that the authors used but is incorrect is the following: if A and B are similar in experiment 1 (A1 and B1) but are different in experiment 2 (A2 and B2), then the outcome of the two experiments is different. This conclusion is obviously not correct. If A1 and A2 are equal to 0 and if B1 is N(1,0.8) and B2 is N(1.4,0.8), then A1 and B1 are not statistically different (p=0.11) whereas A2 and B2 are (p=0.04). Obviously B1 and B2 are not different. This type of argument is used many times throughout the result section.
For instance, the authors used this kind of comparison to assess the difference in generalization between their groups. In Fig. 2.C, actual and expected forces are different whereas they are not in Fig. 4.C. The authors therefore concluded that the patterns of generalization were different in the two experiments. More importantly, the authors used this logic to reconcile their data with the conclusion from Goodbody and Wolpert (Fig.6). However, superposing Fig. 2.C and 7.C clearly shows that the response in the two experiments are similar.
In sum, there is a clear lack of between-group comparisons, which does not allow the reader to assess the actual difference in generalization patterns. Such a difference might be assessed by using functional ANOVA to compare the actual force profiles from Fig. 2.C, 5.C and 6.C (given that expected force profiles are comparable). Finally, to avoid this statistical flaw, the authors should summarize their data by one parameter normalized for speed (see comment #2) and should directly compare their different conditions.
2. In panels E of Fig. 2, 4, 5 and 6, the authors used the difference in total force production as their dependent measure. However, this measure depends on the level of expected force, hence on the speed of the movements, which are different for the two different movement amplitudes. For instance, let’s consider that EF1(t) and EF2(t) are the expected force profiles from panel 2.B and 2.C (15 and 30cm movement amplitudes, respectively) and that the corresponding actual forces are hypothesized to be equal to 0.8*EF1(t) and 0.8*EF2(t). In this case, the total force production (integral of 0.8*EF1(t) and of 0.8*EF2(t), respectively) will differ because EF1 and EF2 are different (given the difference in movement amplitudes). This reasoning shows that the dependent measure chosen by the authors is not comparable across different movement speeds whereas comparing force profiles across different movement amplitudes is the actual goal of the paper.
Therefore, the authors normalize their dependent measure for the level of expected force and at best they use one of the two typical measures that are used to assess the level of adaptation in force-field paradigms, i.e. the percent perturbation or the regression coefficient measures (Scheidt et al. 2000 JNeurophy, Wang and Smith, 2008 JNeurosci). If the authors decided to keep a normalized version of their measures, they should explain the reasons for not using one of the standardized measures.
In addition, people usually found that lateral force production during channel trials was not equal to the expected force, especially after 150 trials (e.g. Scheidt et al. 2000, Wagner and Smith 2008) but quickly reached 80% of the expected force. Therefore, testing whether actual and expected forces are similar does not seem to be adequate.
3. The authors claimed that when there is an increase in movement amplitude, subjects only generalized when the hand velocity was within the range of velocities experienced during training (p.431, right column, second paragraph). I’m still looking for one statistical analysis supporting this claim…
4. In this study, increasing the amplitude led to a corresponding increase in velocity, which, the authors hypothesized, impaired the ability to generalize. If this hypothesis is correct, increasing the amplitude of the movements without increasing their velocity should lead to full generalization (OK given Goodbody and Wolpert). On the other hand, increasing the velocity without changing the amplitude should lead to absence of generalization (opposite to Goodbody and Wolpert). To confirm their theory, the authors should run those two additional experiments.
5. The title of the paper is misleading as the authors studied generalization across different amplitudes AND across different velocities at the same time.
6. The authors should clearly define what they call complete generalization or absence of generalization. It seems to me that the authors did not consider any possibility between those two. However, it has clearly been shown that directional generalization is not on or off but that there is partial generalization for neighboring directions. Fig. 8 from this paper also suggests that there is some generalization going on when the amplitude of the movements are increased.
Conclusion
In sum, given all the flaws above, this paper is really inconclusive with respect to whether there is full or partial generalization across different movement amplitudes and speeds.
Agree or Disagree
9 
0