March 12, 2018 at 5:38 pm #21518
I performed a structural connectomic analysis with Braph and I found a strange results that you can visualize at the link below:
As you can see while on the left (in the table with the nodal measures)the p(2tailed) value at density 5 is significant (1.000e-03), on the right the circle indicating the difference between groups falls within the CI.
Could you, please, help me to understand these incongruent results?
Thanks in advance
AnnabellaMarch 17, 2018 at 8:08 am #21548
thank you for your asking this. It’s in fact strange. It might be a bug in the code.
Could you share with me the .mga file with these data by email so that I can see what is the problem?March 19, 2018 at 3:38 pm #21555
I had a look at the results you sent and I think I understand this issue. It arises due to the way that the results from multiple comparisons are presented in the table and plot in BRAPH. Therefore, let me try to explain them more in detail, hopefully this will help.
In the table, in the columns titled p(1-tailed) and p(2-tailed), BRAPH shows the p-values from single hypothesis testing. These raw p-values (the 1-tailed one) are the values from which the confidence interval is derived; as such, they are the values reflected in the plot. In your example, these p-values are not significant (>0.05), and as a result the difference value lies within the confidence interval. Actually, the last single tailed p-value is small, thus, the difference is outside of the interval as expected.
Due to the problem of multiple hypothesis testing, the issue of statistical significance cannot be deduced from these p-values and they need to be additionally corrected. In BRAPH, this is implemented by controlling for the false discovery rate. To calculate the false discovery rate (corrected by using the Benjamini-Hochberg procedure, http://braph.org/manual/brain-graphs/) the p-values are ranked in ascending order and compared with their false corrected values. Once the largest p-value that is smaller than the corresponding false-rate-corrected value is identified, all the p-values smaller than this value are considered significant. These multiple comparisons are performed at a particular density for a given measure across all of the regions in the brain atlas.
So, the fdr part of the table works in the following way: The value in the fdr column is the largest p-value that is smaller than the corresponding false-rate-corrected value as mentioned above (this value can correspond to any region, but it is calculated at the particular density). Therefore, if the p-value that is shown in the p-value column is smaller than the corresponding fdr value shown in the fdr column, that region is significant at this density. Conversely, if you have a zero in the fdr column this means that none of the regions show a significant differences at that particular density value once their corresponding p-values are fdr corrected (since p-value cannot be negative). Please note that as you change the regions from the popup menu, the fdr value does not change since for a given measure it depends only on the density.
If you would like to visualize all regions that pass the fdr corrections at a given level, you can use the Brain View part of the same interface (MRI Graph Analysis BUD).
I hope that this clear up the issue you had. If you need further help, please do not hesitate to contact me.