Principles of how students learn (from Donovan and Bransford (editors). 2005. How Students Learn....)

Notes to myself:

The introductory chapter of this NRC book summarizes an earlier NRC report How People Learn: Brain, Mind, Experience and School.

They describe three key principles:
1. New knowledge must connect to existing knowledge already learned.
2. Facts and conceptual framework go together, hand-in-hand. A framework with facts is relatively meaningless (an empty framework) and facts without a framework make no sense, cannot be retained, or recalled.
3. Metacognition (understanding tips, tricks, and principles of learning) helps facilitate learning.

One common trap that I fall into is that I fail to appreciate what a limited experience most students students have of the natural world. Therefore, I fail to connect to their existing knowledge base. To connect this to the principles above, I fail to give students enough facts for a new conceptual framework. I assume that they already have lots of facts in hand (what a maple tree looks like, or what a sow bug acts like). What I may want to do is say or ask:

1. "Here is a new conceptual framework, and here is how it works and what it is good for."
2. "Here is a specific example of an empirical experiment that helped confirm the utility of this framework. This is how this example fits into this framework."
3. "Here is another example...can you figure out how this example fits into the framework?"
4. "Here are more examples. Go for it."
5. "Can you find other examples?"
6. "Can you imagine other ways to investigate the natural world using this framework?"
7. "What do you like about this framework? What do you find confusing or frustrating about this framework?"
8. "How might you modify this framework?"

(I might not get around to #7)

Reflections on mathematical modeling (II)

Brain data dump...

levels of formalization:

• what do previous data tell us - deterministic models (e.g., average, linear regression)
• what do previous data tell us - stochastic models (e.g., range, standard dev., standard error)
• increased sophistication (e.g., non-normal forms of stochasticity: null models, interesting parametric distributions).
  
• meta-analysis - combining previous empirical studies
  
• models with and without feedback or loops

Learning a language, learning modeling concepts.

Modeling data, modeling dynamics.

Learning by,

1. copying,
2. applying,
3. combining,
4. creating.
   

Discussing scientific papers in classes - what do we DO?

Should we demonstrate understanding during class time, or should we just jump ahead? I think we need to demonstrate understanding in class, if only to make sure people actually work at reading the assigned papers. However, we could even read the paper out loud, but that would not guarantee understanding. So, it seems to me that in each class we should address at least the following questions:
1. Is the question addressed in the paper interesting?
2. Do the data address the hypotheses?
3. Do the results support the conclusions?
4. What are the implications of the conclusions (or of the results)?

In class, we might start with #2, then #3, #4, and then maybe return to #1.

Pedagogical and scientific goals

I posit that understanding is the core value of mathematical modeling. There are (at least) two levels of understanding, the understanding of our own questions. The first aspect of understanding enhanced by modeling is making our spoken language precise with the aid of mathematics. The second aspect of understanding is providing an unambiguous structure to our ideas that the scientific community can use, that is, the development of useful theory.

I like to think of the scientific process of knowledge creation as a 3D spring, coil, or spiral, where a single loop represents a complete cycle of the scientific process (question, hypothesis, test, interpretation), and progress occurs as we repeat the process through multiple cycles, traveling down length of the coils. Mathematical modeling can help us at different phases of a single coil.

I think that making ourselves formalize our conceptual models helps us see and understand our ideas to a greater degree. Formalization helps us become ever more specific and thereby operationalize our hypotheses and thereby generate more testable predictions. Going through the formalization process helps us understand what a mathematical model is and and how mathematical models provide structure to theory. The process helps show us and convince us of how models are used in Science.

Reflection on teaching modeling, or why should non-modelers try to model?

At the moment, I believe that non-modelers (students or faculty) benefit from attempting to model simple systems. I believe that it helps them become better scientists.

I am near the end of a semester in which we tried to incorporate a little bit of modeling into an otherwise basic graduate level ecosystems course. I think I would like to reflect a bit.

For years I have helped teach a population/community grad course, where we included basic population and food web models, and a smidgen of other stuff. In that course, we started everyone out making the same assumption of ignorance for all, and we taught just enough for students to implement simple models in R. I am not sure how satisfactory it is. I think I want to teach more basic R so that students learn about R in a modeling context, not just their stats classes. I think by learning R they will learn about models even more effectively.

This semester (Winter/Spring 2011), in the ecosystems course, we started students thinking along two tracks, one of conceptual models of ecosystems and the other learning the R language. Our thought was that by the time they had learned enough about ecosystems, to create conceptual models, they would have learned enough R to begin formalizing their conceptual models. However, that has not been the case, for at least two reasons.

The first cause of sub-optimal pedagogy may have been that students new to a language (e.g., R) need to work with it at least three days/week (preferably 4-6), but I did not structure the assignments that way. They need both carrots and sticks, and assignments that require daily turn-around (e.g., automated release and deadlines, or email with 24 hours to upload answers). I would not even have to grade every one of them - just mark them turned in or not, perform spot checks, and provide detailed answers. Why didn't I do this? Several not-very-good reasons:

• I felt sorry for them,
  
• I wasn't 100% convinced that I should push programming and math that hard,
  
• it would have been more work for me,
  
• not everyone needed that kind of practice,
  
• those that needed that kind of practice COULD have done self-study.

The second reason for suboptimal pedagogy was that I tried to be more flexible with the modeling assignments than I was easily capable of -- I could create stuff, but some of it took longer than was convenient. In brief, we asked students to come up with a scientific question, explain what is known and unknown regarding that question and their study system, and design a conceptual model that captures the essence of their question and/or system. Students were then asked to formalize their conceptual model using mathematics or computer code or both. The students conceptual models were not all ecosystem models with merely pools and fluxes of all the same units and element(s). Rather, most were a hodge-podge of different sorts of variables that related typically in a mechanistic fashion, but were not comprised of, for instance, pools and fluxes of carbon. Therefore, the relatively low programming ability of the students (see first reason, above) and my desire to be flexible with regard to acceptable topics meant that I had to invent lots of unique code for each different student. And that, Virignia, is the second reason why my pedagogy was sub-optimal.

However, I think that forcing students to formalize their conceptual models has helped them see and understand their own conceptual models to a greater degree. Formalization helps them become ever more specific with regard to their conceptual model and this helps them generate more testable predictions. Formalization helps them understand what a mathematical model is and and how mathematical models provide structure to theory. The process helps show them how models are used in Science, and last, it helps them see indirect connections more clearly and accurately. Well, ... I hope it does all that.

What is your Bayesian prior on non-human animal emotion?

Although humans have a strong ability to communicate with other humans, we lack the ability to communicate as well with non-human animals. As a result, we know far more about the internal emotional states of other humans than we know about the internal emotional states of non-human animals. Where do we begin to make inferences about the internal states of non-human animals? What is the appropriate Bayesian prior for making inferences about the unknown internal states of non-human animals?

A long time ago, Rene Descartes (1649) posited that non-human animals are akin to machines (have no soul or mind and couldn't feel pain). Since then, it has been considered "scientific" to assume non-human animals actually are machines unless there is overwhelming evidence to the contrary. I propose that this is highly unscientific, and that it is an example of argumentum ad ignorantiam, that the absence of evidence is the evidence of absence. That is, Descartes, and millions since then, have found it convenient to presume that an absence of evidence about non-human animal emotion constitutes evidence of absence of non-human animal emotion.

Another approach to understanding non-human animal emotions, universally taken by infants, is to assume that "all others are like me." Indeed, this is how humans learn about each other. In the absence of a lot of knowledge, we assume that other humans (especially those that look like us) think and feel as we do. This is our Bayesian prior for interactions with other humans. Like Descartes's proclamation that non-humans are machines, this could also constitute a prior for understanding non-human animal emotion.

Is emotion likely to be a synapomorphy shared with a wide range of taxa? In my simplistic way of thinking, I propose two lines of reasoning that suggest it is. First, emotion in humans appears to be regulated in a primitive part of the central nervous system, whose structure is shared with a wide variety of taxa. If it looks like beer, smells like beer, tastes like beer, it might be beer. Second, emotions are useful. Fear and pain are widely recognized for their fitness benefit, for their adaptive value. I suggest that other emotions are likely to have similar fitness benefit. If a behavior generates joy or euphoria or happiness, organisms would be inclined to continue that behavior. Emotions could help provide mechanistic links between biochemistry and behavior.

Bayesian inference is useful in that it requires we think hard and think carefully about our prior beliefs. In that sense, it helps us become more scientific and maybe even more moral.

To P or not to P, that is the question...

I think that at least part of the reason most of us have a hard time stating what a frequentist P-value is (and is not) is because we do not know the difference between statisticians' correct definition and our common but erroneous definitions. In addition, or maybe another way of looking at it, is that the ambiguity of words in general (as opposed to math) contributes substantially to the confusion.

Here I take a stab at it.

A frequentist P-value (say, of a t-test for a difference between means) is the probability that a difference as large or larger than the observed difference would occur if our two samples were drawn from the same distribution, and the same experiment were conducted repeatedly ad infinitum (or ad nauseum). "A P value is often described as the probability of seeing results as or more extreme as those actually observed if the null hypothesis were true" (2).

A frequentist P-value of such a t-test is apparently not lots of things we wish it were (1):

1. it is not the probability that our null hypothesis is true.
2. it is not the probability that such a difference could occur by chance alone.
3. it is not the probability of falsely rejecting the null hypothesis.

Each of these leave information out. Important elements of a correct definition include

• The probability of observing data as extreme or more extreme in ...
• repeated identical experiments and analyses, given ...
• the a priori belief that the null hypothesis is true (this is a Bayesian "prior").

Thus we see that the incorrect definitions typically share something with a complete definition, and that under most circumstances, analyses that fit incorrect definitions (e.g., Bayesian posteriors) will be correlated frequentist P-values (see comment by Bolker comment below).

A frequentist confidence interval is a region calculated from our data and our selected confidence level, α, in a manner which would include the "true" population parameter (e.g., the "true" mean) 100(1-α)% of the time if the same experiment and analysis were conducted repeatedly ad infinitum. It is a region which, so calculated, would include the true population parameter in 95% of all hypothetically repeated identical experiments. Thus, the population parameter of interest is fixed (i.e., a "true" value exists), and the interval is random (because it is based on a randomized experiment).

A frequentist confidence interval is not an interval which we are 100(1-α)% certain contains the true parameter. I don't even know what this statement (i.e., 95% certain) means -- what is "certainty"?

A 95% Bayesian credible interval (a.k.a. Bayesian confidence interval) is the a continuous subset (i.e., an interval) of a posterior probability distribution of a parameter of interest (e.g., a mean). A posterior probability distribution is the probability distribution which results from combining of our prior beliefs about the parameter, and a conditional probability distribution of our data, given all possible relevant data sets. It contains all possible values of our parameter of interest (given our priors and our data, and our model). The credible interval is merely an interval which contains a most likely subset. That is, we are not sure what is was while the world turned and our data were collected, but the safest bet is inside the credible interval.

Key differences between frequentist and Bayesian statistics:

• Parameter of interest (e.g., a mean): fixed (the Platonic archetype exists) vs. random (i.e., subject to the whims of the gods of stochasticity)
• Prior beliefs: implicit and sometimes hard to discern vs. explicit and plainly stated.
• Statements of probability: Pr(data|null) vs. Pr(h|data). That is, frequentist P-values are the probability of observing your data (or more extreme data) given that the null hypothesis is true, whereas Bayes posterior probability distributions describe the probability of your scientific hypothesis (not the null), given that your data are true.
• "P-values": Exist vs. not typically presented (usually misinterpreted; Fisher used it as a "weight of evidence," whereas Neyman used it as a basis to make decisions (yes/no) but not necessarily true/false - I do not understand how or why they can do all that; in the Bayesian context, it is not always clear what such a beast would be because of differences in underlying interpretations).

My problem is that I do not have an intuition about how these things all differ or under what conditions they are likely to differ substantially. Therefore, I cannot keep them clearly differentiated in my head. All I can do is repeat them. It would help to explore the pathological cases where frequentist and Bayesian methods result in very different outcomes differ strongly.

1. http://en.wikipedia.org/wiki/P-value#Frequent_misunderstandings
2. http://www.jerrydallal.com/LHSP/pval.htm