In these cases I personally look whether the question is on
- The interpretation of the results (on-topic), or whether it is on
- Mathematical procedures per se (off topic).
To clarify this with an example; let's look at the linked question:
Say I have a go/go no task, and my output data includes 3 parameters: avg. reaction time, variance of RT, and number of errors. I want to composite all the parameters into a single index score- what is the best way to do it? all the parameters are equally important to me.
Now, let's replace the key parts with abstract symbols:
I have 3 outcome parameters: average, SD, and number of incorrect answers. I want to composite all the parameters into a single index score- what is the best way to do it? all the parameters are equally important to me.
To me, the question doesn't make sense anymore, as the #incorrect answers is a typical outcome parameter for psychophysical tests, which is the domain of psychology. Further the psychophysical go/no-go task is essential to understand the question. In other words, this is all about the interpretation of data in the realm of psychology, and hence on-topic imo.
Now let's take an obvious example that is, imo, offtopic:
I have a population of people with suicidal thoughts. I have given
them a questionnaire with yes/no questions and now I wish to know if
their suicidal thoughts are linked to their having a depression or
not, and whether it's linked to a familial history of suicides.
That's offtopic imo, as it can be replaced by meaningless symbols and reduced to a hardcore stat question:
I have given my subjects with disease X a questionnaire with yes/no questions and now I wish to know if X is linked to 2 specific binary question outcomes?
Answer: You have to use a test based on a binomial distribution, such as....