Stats Question: Social Sciences

[ck926]

Hi everyone,

 

I'm having some trouble figuring out the right analysis to use for a measure validation project. I've gotten a few different answers among faculty members, so I just wanted to get some more opinions. Basically, the measure consists of five different vignettes or "scenarios". The same 9 items are used to respond to each of the scenarios. In other words, participants answer the same set of questions five different times. I've been told to do a factor analysis; would I do five different ones for each vignette? Or should I be taking a totally different approach? Thanks in advance!

[lewin]

You could create a measurement model in AMOS that could account for the dependency in the data (i.e., that each set is nested within scenario). I can't for the life of me remember how to set up EFA in AMOS though, it's more intuitively able to do CFA. Which type are you doing? For CFA I suspect you'd specify the factors as latent variables with paths to the items, and the paths constrained to be equal across scenarios (e.g., item 1 always has the same path estimate, i.e., relationship with factor(s), across the 5 scenarios).

[juilletmercredi]

If I'm understanding this correctly, let's say that the scenarios are all about drug use for adolescents - so one might be a vignette about drinking with friends when parents aren't home, one might be about smoking marijuana when parents aren't home, etc. - and the questions might be something like "How confident are you that you could refuse to do this action with your friends?"; "How tempted would you be to join in?"; "How difficult would it be for you to turn down your friends' offer?", etc. And then maybe your domains/factors are "confidence/self-efficacy," "temptation", "pressure/difficulty," etc. Presumably you'd have one question for each factor asked across the 9 different scenarios - so 5 items for each of 9 factors. (Or maybe some of the items feed into the same factor - it depends).

 

Maybe I'm missing something, but I can't see why you would have to account for dependency in that model, because the fact that your items are nested within scenarios doesn't seem matter. You're not expecting the answers on the 9 different items within each scenario to be more correlated within each other than the answers across the scenarios, because they are tapping into different domains/factors, right? (If you had multiple questions tapping into the same factor within each scenario, then the clustering does potentially become more of an issue. I think most scientists would ignore it, for better or for worse. You could do a multilevel CFA if you wanted, though, just to test and see the differences in your analyses.)

 

I used Stata for a similar problem; specificying CFA there was quite simple. It is set up very much like a latent variable analysis - with the factors as latent variables with paths to each item you think leads to that variable, and paths constrained to be equal. You can insert a term that makes the structure multilevel to control for the correlation within scenarios. Paths constrained to be equal would be another way to solve that problem. You could play with it a bit and see what works. I'm sure that you could do it in AMOS, but I don't use it and I don't know how. If you know R you could probably also use R to do it, and I think SAS would handle it too.

Edited May 6, 2015 by juilletmercredi