Stats Question- BL differences with multiple IVs

[PsychGirl1]

I have a stats question that I just can't seem to answer. My adviser keeps telling me to do one or the other depending what day of the week it is, I've gotten mixed answers from other grad students, and I'm having trouble finding anything about it in my textbooks or online.

 

I have 3 IVs in my thesis, which equates to 8 conditions. Two are randomized conditions, and one is based on a score of a measure. I want to check and see if there are baseline differences between groups- for example, BMI.

 

I've come up with 3 ways to do this, and I don't know which is correct:

 

(1) A one-way ANOVA across the 8 conditions.

(2) Three t-tests across the three IVs.

(3) A 3-way ANOVA with BMI as the DV (my primary analysis is a 3-way ANOVA).

 

I originally thought I should do #1 or maybe #2. My adviser told me I should definitely do #2. The other day, he told me that was wrong and I should do #3, since #3 is the format of my primary analysis (2x2x2 ANOVA), so I should test my BL variable the same way.

 

Has anyone seen this before? Does anyone have a good guess on what the correct option would be? I found some research papers, but they are usually quite vague and say something like "examine whether there were baseline differences between conditions"- however one somewhat detailed paper I found referred to their results with F, suggesting they did option 1 or 3.

 

Thank you all!! I appreciate your help.

[lhommependu]

you should do 3.

 

i think 1 will give you the same omnibus F as 3, or a very similar one, but it won't give you any main effects or interactions if you're planning on picking any observed significance apart. 

 

what you're looking for in 2 will also show up in 3, but you'll save yourself some type I error since you're only doing one analysis

[Guest |||]

You risk statistical bias with 2 and, depending on your power, reviewers might not take kindly to such results.

 

Do 3.

[PsychGirl1]

Thank you for the replies! Agree about #2, then that one is off the list :-).

 

What is the reasoning for doing #3 over #1? Since it's something like BMI, you wouldn't expect it to be influenced by experimental manipulations, so why would you care about the interaction terms?

[lewin]

I like #3 for its simplicity but I can see your argument for #1 where you don't need the interactions if it's implausible that the treatments would have affected BMI (assuming it's premeasured). It sounds like #2 doesn't actually test what you want anyway (i.e., making sure that the 8 cells are roughly equal).

 

 

A different question I have for you is why you're treating the measured variable as two conditions. Depending on how you've done this, you're losing a lot of power. If you want to treat it as a continuous variable then this is a classic book that describes how to do the analysis using multiple regression. I'm happy to elaborate if you like.

Edited May 18, 2013 by lewin00

[PsychGirl1]

Thank you for the feedback! Okay, so it sounds like I can have reasons to do it either #1 or #3.

 

I definitely agree with you about treating my measured variable as two conditions. However, this is how we designed it for my thesis, as I'm on really tight timelines and for ease of analysis as well as interpretation, my committee and I agreed that this would be simplest. I already have a really complex thesis, with 3 IVs and 3 DVs (which is reality is 8 DVs, as some of them have multiple variables within them), and set up and ran participants with a 90-minute visit.

 

That being said, when I write it up for a manuscript, I'll be running the analyses either as an ANCOVA (using the continuous measure as the "covariate") or as a regression.

[PsychGirl1]

(BTW I'm defending in less than 2 weeks and I'm still working on the results section.. if that gives you any indication of how tight my timelines are haha. DYING).

[lewin]

Yikes! Yeah sorry, just do the easiest thing acceptable to your committee then. Good luck!

 

[PsychGirl1]

Thank you!!!! I just hope to get through it with some of my dignity intact, but it's not looking too good ;-).

 

I appreciate all your feedback- I might be asking you some questions about the regression in a few weeks!! Technically I know how to do it, but I've never attempted to do it on my own data, and I'm sure I'll run into a billion issues that you never learn about in class... as always.