If you could design an introductory stats course....

[L83Ste]

I'm a TA for stats and looking into Ed. Psych programs. One thing I have noticed after being on the other side of academics is that students are primarily performance-focused rather than learning-focused. Our system has made students this way considering we must get a certain GPA to be rewarded or we must have a certain GRE score and GPA to get into graduate school. We have to have certain grades to pass the courses, so students tend to focus on that rather than learning. They want extra credit to increase their grades, not their knowledge. 

 

I've been reading a book about educational psychology and I'm curious to know what types of ways we can change things to make students more focused on learning rather than grades? Right now, tests in the class I TA for make up 60% of the overall score. 40% are section tests and 20% is the final exam. No wonder students focus on grades on the tests! Anyway, how would you all attempt to change the direction of the focus? What would you do if you designed an intro to stats course (or any course for that matter) to focus on learning rather than grades?

 

 

[Chubberubber]

First of all- if only 60% of the final grade is determined by tests, then it's a very good start. What is the other 40% determined by? 

Seems to me like in order to do well (meaning- more than just passing), a student in your class can't focus solely on the tests. But, to answer your question- I would have the entire grade be based on weekley statistical analysis projects combining whatever was learned in the past class with everything that was alreadt taught. I would phrase the question in a way that wouldn't tell the students which statistical procedure to carry out, but rather give them a question and see if they can understand themselves which procedure is needed and how to carry it out,

[CogPsych2015]

In the Methods course in my department, a solid chunk of the grade is devoted to a cumulative research project that gets presented at the end of the semester. There are too many students in the course to gather "real" data, but students come up with an idea for a project, make up stats, and then present it in research-poster format. I think it was a great chance to really show you learned something throughout the course. You had to use information from week 1 (what are IVs vs. DVs, operational definitions, etc.) all the way through factorial design and ethics. It was a whole heck of a lot of work, but I knew I was actually being assessed on what I'd learned, not what I'd memorized for an exam.

[L83Ste]

First of all- if only 60% of the final grade is determined by tests, then it's a very good start. What is the other 40% determined by? 

Seems to me like in order to do well (meaning- more than just passing), a student in your class can't focus solely on the tests. But, to answer your question- I would have the entire grade be based on weekley statistical analysis projects combining whatever was learned in the past class with everything that was alreadt taught. I would phrase the question in a way that wouldn't tell the students which statistical procedure to carry out, but rather give them a question and see if they can understand themselves which procedure is needed and how to carry it out,

The other 40% is determined by homework (30%) and participation (10%).

 

In the Methods course in my department, a solid chunk of the grade is devoted to a cumulative research project that gets presented at the end of the semester. There are too many students in the course to gather "real" data, but students come up with an idea for a project, make up stats, and then present it in research-poster format. I think it was a great chance to really show you learned something throughout the course. You had to use information from week 1 (what are IVs vs. DVs, operational definitions, etc.) all the way through factorial design and ethics. It was a whole heck of a lot of work, but I knew I was actually being assessed on what I'd learned, not what I'd memorized for an exam.

Now, I didn't design the course, but I wonder if giving the students each a project to start and letting them work on it from the very beginning the entire semester would show how far they had come. They would turn in a portfolio of all of the analyses run on the original data and all of their work leading up to the final submission. This would be a ton of work, but I think it would really benefit students. I would have loved doing that. 

[mb712]

In the intro class I took, only 30% of the grade was exams. The rest were analyses and write ups, SPSS usage tests, that kind of stuff. We also had a semester long project, similar to what you mentioned above, which we turned into a poster to present and mock journal submission. I LOVED the class.

[L83Ste]

In the intro class I took, only 30% of the grade was exams. The rest were analyses and write ups, SPSS usage tests, that kind of stuff. We also had a semester long project, similar to what you mentioned above, which we turned into a poster to present and mock journal submission. I LOVED the class.

I think this sounds more beneficial than 60% exams. Doing 60% exams which consist of multiple choice and calculation seem to put the focus on grades IMO. We actually could have students present posters at our student research day on campus if we did it that way. SPSS assignments are added to the homework, so they get some experience with it. They have to make up their own data sets and write explanations of the analyses. I would love to hear more about the types of assignments, exams, usage tests, write ups, and the project that was required! 

[TenaciousBushLeaper]

If I could design an intro stats course, I wouldn't have any exams. All assignments would be more like short 1-2 page papers where students would have to work through problems to show their knowledge of the material. Students could re-do these assignments for better grades, provided they've made a significant improvement on the assignment. In a perfect world, students would discuss concepts in class (the professor/ta would lead the lecture/discussion but students would need to have mandatory discussion questions) & then during lab students would apply the concepts from the lectures. 

Maybe a mock up assignment would be, given a data set students would need to analyze the data based on a target goal set by the instructor. On top of accurately applying the concepts, students would need to justify the type of analysis. Why they choose analysis x over analysis y, what information does it provide, what are the drawbacks.  

[psych face]

I think exams, constructed properly, can accurately assess learning. Although, the Ed. Psych dude here has eliminated exams entirely from his courses, so take that for what you will. He uses interactive learning, group work, homework exercises, and generally forces the students to constantly prove they know the material through their experiences in class. That's all well and good, but I think it would be really difficult for the instructor to actually assign grades in that atmosphere. Now, if we didn't have to assign grades, that would be great. It would just be a pass/fail situation for the course, and you would only get credit if the professor considered you competent in the material. But this is more like a conceptual redesign of the entire system, and I don't think we're prepared for that right now, what with everyone's dependence on grades.

But, Ideally, I think it would be better. However, I would still incorporate exams to verify basic acquisition of simple things like terminology necessary to sound like an educated adult moving forward. For example... I met someone once who was taking senior level biology courses and didn't even know how many chromosomes there were and thought all your heritable genes were on the X and Y chromosomes. I wanted to vomit, that the guy was getting a degree in biology. That's the sort of thing exams are for... People who fail basic stuff like that should be prevented from graduating. I mean, I think he slipped through the cracks solely as a result of these more 'fluid' experiential grading systems a lot of professors here are using now. A course based solely on exams would have prevented that level of incompetency.

I would probably try to find a healthy mix.

But this is all relative to what you are teaching and what your goals are; vocab and concepts vs mechanistic understanding.

I did recently have this conversation with a statistics teacher and he decided to redesign his course in order to have the students following the scientific method, engaging in hypothesis testing. Then the course content, statistical methods, would come as a byproduct of this. Honestly, I'm a little concerned because a lot can go wrong when you try new things out, but his method sounds more practical. It just seems to me the difference between algebra via lists of problems or via story problems. Which, I don't know which is better, so it's tough.

This is constantly being talked about at my school, by the way. Big issue, how to teach stuff.

[isilya]

If I could design an intro stats course, I wouldn't have any exams. All assignments would be more like short 1-2 page papers where students would have to work through problems to show their knowledge of the material. Students could re-do these assignments for better grades, provided they've made a significant improvement on the assignment. In a perfect world, students would discuss concepts in class (the professor/ta would lead the lecture/discussion but students would need to have mandatory discussion questions) & then during lab students would apply the concepts from the lectures.

Maybe a mock up assignment would be, given a data set students would need to analyze the data based on a target goal set by the instructor. On top of accurately applying the concepts, students would need to justify the type of analysis. Why they choose analysis x over analysis y, what information does it provide, what are the drawbacks.

I love that idea! After all, that's what you do when you analyze data... Identify goals, and determine what analysis is best to do given your data and goals.

[suzieq]

If I could design an intro stats course, I wouldn't have any exams. All assignments would be more like short 1-2 page papers where students would have to work through problems to show their knowledge of the material. Students could re-do these assignments for better grades, provided they've made a significant improvement on the assignment. In a perfect world, students would discuss concepts in class (the professor/ta would lead the lecture/discussion but students would need to have mandatory discussion questions) & then during lab students would apply the concepts from the lectures. 

Maybe a mock up assignment would be, given a data set students would need to analyze the data based on a target goal set by the instructor. On top of accurately applying the concepts, students would need to justify the type of analysis. Why they choose analysis x over analysis y, what information does it provide, what are the drawbacks.  

 

I also like this idea, because knowing what analysis to run requires a lot of knowledge.

 

Another thinking point: Many of the students in an intro course will likely NOT really need in-depth stats in their careers. For many, it will be good if they understand percentages and correlations, and a lot of them know that. It can be motivating to use quirky or creative examples that illustrate stats concepts without getting too much into theory. I know that sounds disappointing for those that really care about research, but in an intro class it's important to remember that most of these people won't end up as researchers, and those that will might not even have that career path on their radar yet.

[FinallyAccepted]

I also like this idea, because knowing what analysis to run requires a lot of knowledge.

 

Another thinking point: Many of the students in an intro course will likely NOT really need in-depth stats in their careers. For many, it will be good if they understand percentages and correlations, and a lot of them know that. It can be motivating to use quirky or creative examples that illustrate stats concepts without getting too much into theory. I know that sounds disappointing for those that really care about research, but in an intro class it's important to remember that most of these people won't end up as researchers, and those that will might not even have that career path on their radar yet.

This is true, but also the reason that I graduated with a Masters in General Psychology without the best background in stats. The professor, because the majority of students were in the Counseling Psychology track (and unlikely to need in-depth stats), decided not to teach us SPSS or any stats program. We only went through the most basic of statistics and were given an easy take-home final exam. I ended up with 100% in the class but had to teach myself statistics when it came to running the data that I collected in my quantitative masters project.

The other students were psyched, but I'm less grade-focused, and would have preferred to have learned something. Plus, is it too much to ask that my own psychologist take a course in statistics so they understand how to read and understand journal articles? Take a class that forces you to persevere in something that is difficult?

I don't necessarily think testing situations are our enemy, but we could be using better strategies to help ourselves learn things so that we actually understand them. Assignments which force you to practice running stats and justify one analysis over another would seem to be helpful for that.

[L83Ste]

What are all your thoughts about making students do calculations by hand? 

[CogPsych2015]

What are all your thoughts about making students do calculations by hand?

At an introductory level, I don't think it's necessary. There are so many things to teach in an intro class, it would be hard to find the time. I'm in an advanced stats class now, and even there we don't. He shows us how the calculations are done, which I think is helpful for theoretical knowledge, but we don't actually have to do them.

[Chubberubber]

I had to calculate everything by hand during the first 2 statistics courses I took (intro to stats and statistic methods for psychological research), and hated every minute of it! I got so many answers wrong becuase I had a stupid miscalculation (like switching plus and minus or + and * -that happens to me a lot!). Not to mention that when you get to the slightly more advanced stuff you have to have tons of pages with endless tables in order to do things like t-tets, f-tests etc. I see no reason to force students to calculate things by hand unless they'd have to do it in the final exam...

Edited February 15, 2015 by Chubberubber

[L83Ste]

Honestly, I think they should know the steps and see it done by hand, maybe a few small calculations, but in the grand scheme of things a person will be using a program for all of that and only analyzing the results. It just seems tedious to me to do a ton of calculating when, in real-life situations, they will not be calculating anything by hand. Knowing how the numbers are arrived at by the program is important. Understanding which type of analysis to do is important. Knowing how to interpret results are important, but doing hand calculations don't seem as important to me. 

[spunky]

I’ve both TAed and taught undergraduate statistics/methods courses (and a couple of graduate-level courses) and have tried to experiment with every possible method (even early-childhood education ones) to convey the material in the most efficient way. There are two things that, for better or worse, I have concluded: (a) there has to be an exam component to it and (b ) the exam component has to carry enough weight on the student’s grades that will motivate them to study and review the material.

 

When I started my MA and began taking the same courses as everybody else I was 100% against exams and complained to the instructor (in a friendly manner, of course) about this. My reasoning was that if you had made it this far in your education, you were willing to study and learn the material because you knew it was important. What the prof said was that you always had to assume students will try to get away with doing as little as possible in classes like this because they tend to not like the material. Fast-forward a few years when I started teaching and I did start finding out that students had a harder time mastering the actual concepts behind statistics if they felt they were not going to be tested. It was like they did not need to put on the effort as much to struggle with the concepts and learn the material… and that is an extremely critical thing that needs to happen if you are seeing this stuff for the first time.

 

When I attempted assignment-only courses, my students became incredibly skilled SPSS-button-pushers and that’s about it. Then they would show up at my door a few months down the line when they had to work on their theses/dissertations/manuscripts and I would get very frustrated because they couldn’t even work out the simplest things by themselves. Like I would tell them “we saw this in class, is in your notes, it’s in X or Y chapter of the book”. One of my students actually gave me a very good hint as for why they were unable to apply this stuff: he said that he always worked in a group with other two people or compared his answers with other students to make sure everything was right before handing it in. So the end result was a class where everybody got an A but only 3 or 4 people knew how things worked.

 

Ever since then I decided exams are the only way I had to ensure that people are actually going to try things at home, practice them, struggle with them and, one way or another, learn them. I have changed the focus into making the classes more interactive though… like I use R to code interactive animations of regression or ANOVA (because the entire linear model has a geometric analogue so you can actually show it in pictures), we have class discussions, we have group activities, etc… but I always keep an exam component in my courses now…lurking…waiting. 

 

Honestly, I think they should know the steps and see it done by hand, maybe a few small calculations, but in the grand scheme of things a person will be using a program for all of that and only analyzing the results. It just seems tedious to me to do a ton of calculating when, in real-life situations, they will not be calculating anything by hand. Knowing how the numbers are arrived at by the program is important. Understanding which type of analysis to do is important. Knowing how to interpret results are important, but doing hand calculations don't seem as important to me. 

 

I wholeheartedly agree with this. The prof who taught me linear algebra had this famous (paraphrased) quote saying: "every self-respecting mathematician should, at some point in his or her life, have to find the inverse of a non-trivial matrix BY HAND". And if you have ever had to found the inverse of a matrix by hand then you know the process is both terribly boring and incredibly illuminating. As someone who both teaches and consults statistics I feel the greatest problem that we face is not so much in how we evaluate the material but how we deliver it. I feel a lot of people in Psych and other social scientists can become really good at following “ready-made” numerical recipes and feed them into SPSS but when it comes to actually understanding where these numerical recipes come from and, more importantly, how to adapt them to new types of data or designs well… then all hell breaks loose. 

Edited February 15, 2015 by spunky

[AAdAAm]

I’ve both TAed and taught undergraduate statistics/methods courses (and a couple of graduate-level courses) and have tried to experiment with every possible method (even early-childhood education ones) to convey the material in the most efficient way. There are two things that, for better or worse, I have concluded: (a) there has to be an exam component to it and (b ) the exam component has to carry enough weight on the student’s grades that will motivate them to study and review the material.

 

When I started my MA and began taking the same courses as everybody else I was 100% against exams and complained to the instructor (in a friendly manner, of course) about this. My reasoning was that if you had made it this far in your education, you were willing to study and learn the material because you knew it was important. What the prof said was that you always had to assume students will try to get away with doing as little as possible in classes like this because they tend to not like the material. Fast-forward a few years when I started teaching and I did start finding out that students had a harder time mastering the actual concepts behind statistics if they felt they were not going to be tested. It was like they did not need to put on the effort as much to struggle with the concepts and learn the material… and that is an extremely critical thing that needs to happen if you are seeing this stuff for the first time.

 

When I attempted assignment-only courses, my students became incredibly skilled SPSS-button-pushers and that’s about it. Then they would show up at my door a few months down the line when they had to work on their theses/dissertations/manuscripts and I would get very frustrated because they couldn’t even work out the simplest things by themselves. Like I would tell them “we saw this in class, is in your notes, it’s in X or Y chapter of the book”. One of my students actually gave me a very good hint as for why they were unable to apply this stuff: he said that he always worked in a group with other two people or compared his answers with other students to make sure everything was right before handing it in. So the end result was a class where everybody got an A but only 3 or 4 people knew how things worked.

 

Ever since then I decided exams are the only way I had to ensure that people are actually going to try things at home, practice them, struggle with them and, one way or another, learn them. I have changed the focus into making the classes more interactive though… like I use R to code interactive animations of regression or ANOVA (because the entire linear model has a geometric analogue so you can actually show it in pictures), we have class discussions, we have group activities, etc… but I always keep an exam component in my courses now…lurking…waiting. 

 

 

I wholeheartedly agree with this. The prof who taught me linear algebra had this famous (paraphrased) quote saying: "every self-respecting mathematician should, at some point in his or her life, have to find the inverse of a non-trivial matrix BY HAND". And if you have ever had to found the inverse of a matrix by hand then you know the process is both terribly boring and incredibly illuminating. As someone who both teaches and consults statistics I feel the greatest problem that we face is not so much in how we evaluate the material but how we deliver it. I feel a lot of people in Psych and other social scientists can become really good at following “ready-made” numerical recipes and feed them into SPSS but when it comes to actually understanding where these numerical recipes come from and, more importantly, how to adapt them to new types of data or designs well… then all hell breaks loose. 

 

 

I commend you on changing the focus on the question from "what is the best way to evaluate students" to "what is the best way teach Statistics" because this is really the crux of the issue (at least from an educational standpoint). I feel that if you can teach the content of the course effectively, then it does not matter whether you use exams, presentations, homework, assignments, etc. to evaluate students, students will get good grades because they understand the material they are learning as opposed to regurgitating it in an exam or following a step-by-step process in an assignment. I do feel like many of the introductory courses in research methods I took did not really prepare me well to do research. They seemed more like a workshops devoted to explaining which SPSS windows to click on, how to read SPSS output and how to follow a 'decision tree' of statistical techniques. 

 

You mention that you use R (I feel like I need to jump into that bandwagon ASAP) to create animations and class discussions/activities to teach Statistics. What do you feel you are doing differently than say a standard intro to a research methods course?  I mean like one in which there is a Prof in front delivering a lecture and everybody else is taking notes. Do you feel it is working?

[suzieq]

This is true, but also the reason that I graduated with a Masters in General Psychology without the best background in stats. The professor, because the majority of students were in the Counseling Psychology track (and unlikely to need in-depth stats), decided not to teach us SPSS or any stats program. We only went through the most basic of statistics and were given an easy take-home final exam. I ended up with 100% in the class but had to teach myself statistics when it came to running the data that I collected in my quantitative masters project.

The other students were psyched, but I'm less grade-focused, and would have preferred to have learned something. Plus, is it too much to ask that my own psychologist take a course in statistics so they understand how to read and understand journal articles? Take a class that forces you to persevere in something that is difficult?

I don't necessarily think testing situations are our enemy, but we could be using better strategies to help ourselves learn things so that we actually understand them. Assignments which force you to practice running stats and justify one analysis over another would seem to be helpful for that.

 

It's possible to use creative everyday examples without dumbing down the stats. I'm suggesting that everyday examples that don't require knowledge about complex psychological theories can help to motivate students that don't plan to pursue research careers. High caliber stats knowledge without getting into the weeds of psychological theory. In an undergrad course that I helped to teach, the students had to learn a lot about professors' research and theoretical orientations to be able to follow the class. It made it harder for them to learn the stats, which was the reason that we were there.

[FinallyAccepted]

In an undergrad course that I helped to teach, the students had to learn a lot about professors' research and theoretical orientations to be able to follow the class. It made it harder for them to learn the stats, which was the reason that we were there.

 

That's unfortunate, because the point should be to understand the stats regardless of the specific context.

[juilletmercredi]

I think stats students should have to do hand calculations.  My entire first stats class was all hand calculations; we didn't start using computer applications until the second course.  The hand calculations let you know what the computer is doing - it becomes less "magic" and more math and logic.  I notice that when I served as a TA for classes that didn't require hand calculations, the students leaned more on the "magical thinking" - they had no idea what the program was doing, and thus, they did not know how to interpret the results.  As I tell my students, I can teach a monkey to press the right buttons on the computer and print me some output.  The important part is if they can look at the results and tell me what it means.  I feel like doing the hand calculations - and thus realizing that the regression literally plots a y = mx + b type line, for example, or that ANOVA is simply when the within-group differences are bigger than the between-group ones - prepares students for that step.

 

I also think that statistics taught in a psychology department should require a basic level of psychological theory and understanding.  I think that's why intro psych is a prerequisite for the classes.  I wouldn't expect the students to know my research area, and all the examples wouldn't be based only on my area, but they do need to have a basic understanding of psychological principles because we're applying the stats to those principles - that's the whole point.  Even if students aren't going into research careers.  Psychology and a knowledge of psychological theories and principles is useful outside of academia, too.

 

Personally, I would make a stats class heavy on homework problems, critical thinking exercises and an analytical project.  I wouldn't jettison tests completely, but they'd be weighted probably more around 30%.

 

I agree with spunky.  IN consulting, one of the biggest problems people would have is that they knew how to run and interpret the basic-level analyses but when something was a little different or went off the rails a bit, they got lost.  Part of it is because they didn't understand the theoretical underpinnings of the statistical tests they were doing, so they didn't know how to adjust their approach.

[spunky]

You mention that you use R (I feel like I need to jump into that bandwagon ASAP) to create animations and class discussions/activities to teach Statistics. What do you feel you are doing differently than say a standard intro to a research methods course?  I mean like one in which there is a Prof in front delivering a lecture and everybody else is taking notes. Do you feel it is working?

 

Well, there is certainly room for the traditional lecture-style approach in these classes and I think that’s a very appropriate model of teaching in the beginning because… well, we all have to start somewhere, right? But then I try to make things as interesting and interactive as I can.

 

For example:

 

- I use the rgl package from R a lot to make pretty plots like this one here:

https://www.youtube.com/watch?v=JaMgi4XBjo8

 

I realize that not many people have worked with 3D objects before and I can see it definitely helps people make the connections between simple and multiple linear regression. Also, the General Linear Model can be represented beautifully in terms of planes, vectors and angles. I have found that if you connect ANOVA and regression to their geometric analogues people somehow ‘get it’ more. And then I ask them questions like “what would you think it will happen if I add a lot of data points towards the middle of the regression plane? Will it move? Will it tilt? etc. People vote in their answers with a system like the clickers in undergrad and then I get an immediate impression of whether the whole class is on the same page or not. And because everything is in R I can code right on the spot whichever questions or doubts my students could have and visually demonstrate the answer

 

- I do a lot of in-class group activities that help connect the interpretation of software output with research questions. Like I give them a research scenario and some SPSS or R output and let them come to conclusions of whether the output gives them the answer they want or if something is missing or, sometimes, I even give them output completely unrelated to the question and they can spot it right away! (I, of course, let them know in advance that this is a possibility).

 

- There are some in-class discussions that have got me in some hot water before but I've found that students love them (and my advisor has my back on this). I like discussing with my students the methodology of published but poorly-designed articles. In order to make them interested I (purposefully) choose controversial subjects that will not sit well with my moderate-to-extremely liberal, non-religious, socialist-leaning crowd of students… so stuff that supposedly “shows” that children of gay parents are worse off than children of straight parents or articles that say certain ethnic group or gender is somehow inferior to others. But they key here (and I make that VERY clear from the beginning) is that I don’t want to hear any discussion about the theoretical standpoint of the authors or ethics or anything like that. It’s all about methods methods methods and only methods. Were their statistics done correctly? Do the conclusions from the analysis follow from the analysis itself? What kind of data did they gather? What kind of design did they use? Was the design suitable to the research question?

 

I want to communicate to my students the power that being a good methodologist can convey. This is the one and only area where you can make or break any scientist’s theory or research without having to know anything about his/her area.  You just need to look at their methods/analysis and if that’s flimsy, everything else collapses regardless of how much BS they want to throw at you. I believe once people realize that research methodology/statistics is just critical thinking on steroids they become much more aware of the importance of this seemingly dry and dull area has.

 

The best compliment I ever got was that, towards the end of the semester, one student told me that after taking my class she felt like a door had opened inside her mind and now everything she did was question everything…. Hidden assumptions, bad data practice… it’s all everywhere! It’s almost like when Neo from The Matrix “wakes up” and he becomes able to see the computer code all around him… things become apparent for what they really are and not what they intend to be. 

[psych face]

I want to communicate to my students the power that being a good methodologist can convey. This is the one and only area where you can make or break any scientist’s theory or research without having to know anything about his/her area.  You just need to look at their methods/analysis and if that’s flimsy, everything else collapses regardless of how much BS they want to throw at you. I believe once people realize that research methodology/statistics is just critical thinking on steroids they become much more aware of the importance of this seemingly dry and dull area has.

 

 

I like a lot of what you have to say but I think it's also important to convey to students that if they do find they have the ability to break the methodology of someone's research, that it does not necessarily mean that the underlying theory is false, it just means that this particular attempt to prove the theory was a failure. The theory may still be workable. That's a problem I've come across a lot; sometimes they think one failure of an attempt means failure of the underlying theory, when really it was just the researcher's attempt that was a failure. Incorrect assumptions might have been made between the initial theory and the methodology, so the results don't imply what the researcher believes they do. Might be off topic, but also a difficult thing to teach sometimes.

Edited February 17, 2015 by psych face