In today’s article, we are going to discuss Pearson Residuals. A Pearson Residual is a product of post hoc analysis. These values can be utilized to further assess Pearson’s Chi-Square Test results.
If you are un-familiar with The Pearson’s Chi-Square Test, or what post hoc analysis typically entails, I would encourage you to do further research prior to proceeding.
Example:
To demonstrate this post hoc technique, we will utilize a prior article’s example:
The "Smoking : Obesity" Pearson’s Chi-Squared Test Demonstration.
# To test for goodness of fit #
Model <-matrix(c(5, 1, 2, 2),
nrow = 2,
dimnames = list("Smoker" = c("Yes", "No"),
"Obese" = c("Yes", "No")))
# To run the chi-square test #
# 'correct = FALSE' disables the Yates’ continuity correction #
From the output provided, we can easily conclude that our results were not significant.
However, let’s delve a bit deeper into our findings.
First, let’s take a look at the matrix of the model.
Model
Obese Smoker Yes No Yes 5 2 No 1 2
Now, let’s take a look at the expected model values.
chi.result <- chisq.test(Model, correct = FALSE)
chi.result$expected
Obese Smoker Yes No Yes 4.2 2.8 No 1.8 1.2
What does this mean?
The values above represent the values which we would expect to observe if the observational categories measured, perfectly adhered to the chi-square distribution.
(Karl Pearson)
From the previously derived values, we can derived the Pearson Residual Values.
print(chi.result$residuals)
Obese Smoker Yes No Yes 0.3903600 -0.4780914 No -0.5962848 0.7302967
What we are specifically looking for, as it pertains to the residual output, are values which are greater than +2, or less than -2. If these findings were present in any of the above matrix entries, it would indicate that the model was inappropriately applied given the circumstances of the collected observational data.
The matrix values themselves, in the residual matrix, are the observed categorical values minus the expected values, divided by the square root of the expected values.
Thus: Standard Residual = (Observed Values – Expected Value) / Square Root of Expected Value
The Pearson Residual Values (0.39036…etc.), are an estimate of the raw residual values’ standard deviations. It is for this reason, that any value greater than +2, or less than -2, would indicate a misapplication of the model. Or, at very least, indicate that more observational values ought to be collected prior to the model being applied again.
The Fisher’s Exact Test as a Post Hoc Analysis for The Pearson's Chi-Square Test
Let’s take our example one step further by applying The Fisher’s Exact Test as a method of post hoc analysis.
Why would we do this?
Assuming that our Chi-Square Test findings were significant, we may want to consider a Fisher’s Exact Test as a method to further prove evidence of significance.
A Fisher’s Exact Test is less robust in application as compared to the Chi-Square Test. For this reason, the Fisher’s Exact Test will always yield a lower p-value than its Chi-Square counterpart.
(Sir Ronald Fisher)
fisher.result <- fisher.test(Model)
print(fisher.result$p.value)
[1] 0.5
<Yikes!>
Conclusions
Now that we have considered our analysis every which way, we can state our findings in APA Format.
This would resemble the following:
A chi-square test of proportions was performed to examine the relation of smoking and obesity. The relation between these variables was not found to be significant χ2 (1, N = 10) = 1.27, p > .05.
In investigating the Pearson Residuals produced from the model application, no value was found to be greater than +2, or less than -2. These findings indicate that the model was appropriate given the circumstances of the experimental data.
In order to further confirm our experimental findings, a Fisher’s Exact Test was also performed for post hoc analysis. The results of such indicated a non-significant relationship as it pertains to obesity as determined by individual smoker status: 71% (5/7), compared to individual non-smoker status: 33% (1/3); (p > .05).
In today’s entry, we are going to discuss Cohen’s d, what it is, and when to utilize it. We will also discuss how to appropriately apply the methodology needed to derive this value, through the utilization of the R software package.
(SPSS does not contain the innate functionality necessary to perform this calculation)
Cohen’s d - (What it is):
Cohen’s d is utilized as a method to assess the magnitude of impact as it relates to two sample groups which are subject to differing conditions. For example, if a two sample t-test was being implemented to test a single group which received a drug, against another group which did not receive the drug, then the p-value of this test would determine whether or not the findings were significant.
Cohen’s d would measure the magnitude of the potential impact.
Cohen’s d - (When to use it):
In your statistics class.
You could also utilize this test to perform post-hoc analysis as it relates to the ANOVA model and the Student’s T-Test. However, I have never witnessed the utilization of this test outside of an academic setting.
Cohen’s d – (How to interpret it):
General Interpretation Guidelines:
Greater than or equal to 0.2 = small Greater than or equal to 0.5 = medium Greater than or equal to 0.8 = large
Cohen’s d – (How to state your findings):
The effect size for this analysis (d = x.xx) was found to exceed Cohen’s convention for a [small, medium, large] effect (d = .xx).
Cohen’s d – (How to derive it):
# Within the R-Programming Code Space #
##################################
# length of sample 1 (x) # lenx <- # length of sample 2 (y) # leny <- # mean of sample 1 (x) # meanx <- # mean of sample 2 (y)# meany <- # SD of sample 1 (x) # sdx <- # SD of sample 2 (y) # sdy <-
FIRST WE MUST RUN A TEST IN WHICH COHEN’S d CAN BE APPLIED AS AN APPROPRIATE POST-HOC TEST METHODOLOGY.
Two Sample T-Test
This test is utilized if you randomly sample different sets of items from two separate control groups.
Example:
A scientist creates a chemical which he believes changes the temperature of water. He applies this chemical to water and takes the following measurements:
70, 74, 76, 72, 75, 74, 71, 71
He then measures temperature in samples which the chemical was not applied.
74, 75, 73, 76, 74, 77, 78, 75
Can the scientist conclude, with a 95% confidence interval, that his chemical is in some way altering the temperature of the water?
data: N2 and N1 t = 2.4558, df = 14, p-value = 0.02773 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: 0.3007929 4.4492071 sample estimates: mean of x mean of y 75.250 72.875
# Note: In this case, the 95 percent confidence interval is measuring the difference of the mean values of the samples. #
# An additional option is available when running a two sample t-test, The Welch Two Sample T-Test. To utilize this option while performing a t-test, the "var.equal = TRUE" must be changed to "var.equal = FALSE". The output produced from a Welch Two Sample t-test is slightly more robust and accounts for differing sample sizes. #
From this output we can conclude:
With a p-value of 0.02773 (.0.02773 < .05), and a corresponding t-value of 2.4558, we can state that, at a 95% confidence interval, that the scientist's chemical is altering the temperature of the water.
Application of Cohen’s d
length(N1) # 8 # length(N2) # 8 #
mean(N1) # 72.875 # mean(N2) # 75.25 #
sd(N1) # 2.167124 # sd(N2) # 1.669046 #
# length of sample 1 (x) # lenx <- 8 # length of sample 2 (y) # leny <- 8 # mean of sample 1 (x) # meanx <- 72.875 # mean of sample 2 (y)# meany <- 75.25 # SD of sample 1 (x) # sdx <- 2.167124 # SD of sample 2 (y) # sdy <- 1.669046
The effect size for this analysis (d = 1.23) was found to exceed Cohen’s convention for a large effect (d = .80).
Combining both conclusions, our final written product would resemble:
With a p-value of 0.02773 (.0.02773 < .05), and a corresponding t-value of 2.4558, we can state that, at a 95% confidence interval, that the scientist's chemical is altering the temperature of the water.
The effect size for this analysis (d = 1.23) was found to exceed Cohen’s convention for a large effect (d = .80).
In today’s entry, we are going to briefly review Fisher’s Exact Test, and its appropriate application within the R programming language.
Like the F-Test, Fisher’s Exact Test utilizes the F-Distribution as its primary mechanism of functionality. The F-Distribution being initially derived by Sir. Ronald Fisher.
(The Man)
(The Distribution)
The Fisher’s Exact Test is very similar to The Chi-Squared Test. Both tests are utilized to assess categorical data classifications. The Fisher’s Exact Test was designed specifically for 2x2 contingency sorted data, though, more rows could theoretically be added if necessary. A general rule for application as it relates to selecting the appropriate test for the given circumstances (Fisher’s Exact vs. Chi-Squared), pertains directly to the sample size. If a cell within the contingency table would contain less than 5 observations, a Fisher’s Exact Test would be more appropriate.
The test itself was created for the purpose of studying small observational samples. For this reason, the test is considered to be “conservative”, as compared to The Chi-Squared Test. Or, in layman terms, you are less likely to reject the null hypothesis when utilizing a Fisher’s Exact Test, as the test errs on the side of caution. As previously mentioned, the test was designed for smaller observational series, therefore, its conservative nature is a feature, not an error.
Let’s give it a try in today’s…
Example:
A professor instructs two classes on the subject of Remedial Calculus. He believes, based on a book that he recently completed, that students who consume avocados prior to taking an exam, will generally perform better than students who did not consume avocados prior to taking an exam. To test this hypothesis, the professor has one of classes consume avocados prior to a very difficult pass/fail examination. The other class does not consume avocados, and also completes the same examination. He collects the results of his experiment, which are as follows:
Class 1 (Avocado Consumers)
Pass: 15
Fail: 5
Class 2 (Avocado Abstainers)
Pass: 10
Fail: 15
It is also worth mentioning that professor will be assuming an alpha value of .05.
# The data must first be entered into a matrix #
Model <- matrix(c(15, 10, 5, 15), nrow = 2, ncol=2)
# Let’s examine the matrix to make sure everything was entered correctly #
Model
Console Output:
[,1] [,2] [1,] 15 5 [2,] 10 15
# Now to apply Fisher’s Exact Test #
fisher.test(Model)
Console Output:
Fisher's Exact Test for Count Data
data: Model p-value = 0.03373 alternative hypothesis: true odds ratio is not equal to 1 95 percent confidence interval: 1.063497 20.550173 sample estimates: odds ratio 4.341278
Findings:
Fisher’s Exact Test was applied to our experimental findings for analysis. The results of such indicated a significant relationship as it pertains to avocado consumption and examination success: 75% (15/20), as compared to non-consumption and examination success: 40% (10/25); (p = .03).
If we were to apply the Chi-Squared Test to the same data matrix, we would receive the following output:
# Application of Chi-Squared Test to prior experimental observations #
As you might have expected, the application of the Chi-Squared Test yielded an even smaller p-value! If we were to utilize this test in lieu of The Fisher’s Exact Test, our results would also demonstrate significance.
Even an old data scientist can learn a new trick every once in a while.
Today was such a day.
Imagine my shock, as I spent about two and a half hours trying to get the most basic MS-Excel Functions to correctly execute.
This brings us to today’s example.
I’m not sure if this is now a default option within the latest version of Excel, or why this option would even exist, however, I feel that it is my duty to warn you of its existence.
For the sake this demonstration, we’ll hypothetically assume that you are attempting to write a =COUNTIF function within cell: C2, in order assess the value contained within cell: A2. If we were to drag this formula to the cells beneath: C2, in order to apply the function to cells: C3 and C4, a mis-application occurs, as the value “Car” is not contained within A3 or A4, and yet, the value 1 is returned.
If this “error” arises, it is likely due to the option “Manual” being pre-selected within the “Calculator Options” drop-down menu, which itself, is contained within the “Formulas” ribbon menu. To remedy this situation, change the selection to “Automatic” within the “Calculator Options” drop down.
(Click on image to enlarge)
The result should be the previously expected outcome:
Instead of accidentally and unknowingly encountering this error/feature in a way which is detrimental to your research, I would always recommend checking that “Calculator Options” is set to “Automatic”,prior to beginning your work within the MS-Excel platform.
There may be a more efficient way to perform this function, as simpler functionality exists within other programming languages. However, I have not been able to discover a non “ad-hoc” method for performing this task within SPSS.
We will assume that we are operating within the following data set:
Which possesses the following data labels:
Assuming that all variables are on a similar scale, we could create a new variable by utilizing the code below:
COMPUTE CatSum=MEAN(VarA, VarB, VarC). EXECUTE.
This new variable will be named “CatSum”. This variable will be comprised of the mean of the sum of each variable’s corresponding observational data rows: (“VarA”, “VarB”, “VarC”).
To generate the mean value of our newly created “CatSum” variable, we would execute the following code:
To reiterate what we are accomplishing by performing this task, we are simply generating the mean value of the sum of variables: “VarA”, “VarB”, “VarC”.
Another way to conceptually envision this process, is to imagine that we are placing all of the variables together into a single column:
After which, we are generating the mean value of the column which contains all of the combined variable observational values.
** (Clicking on the any of the images displayed below will enlarge their contents) **
First, we will address the steps necessary to suppress unnecessary and unwanted columns within the SPSS Frequency tables.
The process to enable the MODIFY functionality is rather complicated. However, if you follow the steps below, you too will be able to have beautiful outputs without having to endeavor upon a lengthy manual cleanup process.
Steps Necessary to Enable the MODIFY Command
1. Un-install SPSS.
2. Install the latest version of Python Programming Language (3.x). The executable installer can be found here: www.python.org.
(NOTE: THIS STEP MUST STILL BE ADHERED TO, EVEN IF ANACONDA PYTHON HAS ALREADY BEEN PREVIOUSLY INSTALLED.)
3. Re-install SPSS. During the installation process, be sure to make all of the appropriate selections necessary to install the SPSS Python Libraries.
4. From the top menu within SPSS’s data view, select the menu title “Extensions”, then select the option “Extension Hub”.
5. Within the “Explore” tab of the “Extension Hub” menu, search for “SPSSINC MODIFY TABLES” within the left search bar.
6. Check the box “Get extension” to the right of “SPSSINC_MODIFY_TABLES”, then click “OK”.
7. The next screen should confirm that the installation of the extension has occurred.
Steps Necessary to Utilize the MODIFY Command
We are now prepared to obliterate all of those pesky ‘Percent’ and ‘Cumulative Percent’ tables from existence! In order to achieve this as it applies to all tables within the output section, create and run the following lines of syntax subsequent to frequency table creation.
SPSSINC MODIFY TABLES subtype="Frequencies"
SELECT='Cumulative Percent' 'Percent'
DIMENSION= COLUMNS
PROCESS = ALL HIDE=TRUE
/STYLES APPLYTO=DATACELLS.
Steps Necessary to Remove the top Frequency Rows Which Accompany Frequency Table Output
In order to suppress the creation of the type of table depicted above, you must modify your initial frequency syntax.
Instead of utilizing syntax such as:
FREQUENCIES VARIABLES=Q1 Q2 Q3
/ORDER=ANALYSIS.
You are instead forced to utilize a more verbose syntax:
OMS SELECT ALL /EXCEPTIF SUBTYPES='Frequencies'
/DESTINATION VIEWER=NO.
FREQUENCIES VARIABLES= Q1 Q2 Q3
/ORDER=ANALYSIS.
OMSEND.
Doing such adds lines of code. However, it is worth the effort. At least, in my opinion. As the offset to the trade is peace of mind.
How to Suppress Syntax from Printing within the SPSS Output
In order to suppress syntax from printing within the SPSS Output widow, prior to creating output, follow the steps below.
1. From the top menu within SPSS’s data view, select the menu title “Edit”, then select the option “Options”.
2. Within the subsequent menu, select the tab “Viewer”. Then, remove the check mark located to the left of “Display commands in the log”. Next, click “Apply”.
You are now prepared to create SPSS session output devoid of syntax.
How to Modify the Visual Style of SPSS Table Output
If you’d prefer a different, perhaps more readable SPSS table output, the following steps allow for the modification of such.
1. Create a table within SPSS which complies with the system default output style.
2. Right click on the table within the output, and select the options “Edit Content”, “In Separate Window” within the drop down menu.
3. Selecting “Format”, followed by “Table Looks” from the top menu, presents a new pop-up menu which allows for general table alterations.
As an example, select “ClassicLook” from the “TableLook Files:” menu.
Next, click the right “Edit Look” button, then click the tab “Cell Formats”. Within this submenu, the general background of table cells can be modified. Be sure to click “Apply” before clicking “OK”.
4. To save a custom “Look”, again select “TableLooks” from the “Format” menu. Select “Save Look”, with “<As Displayed>” selected within the right “TableLook Files” menu.
5. To load this look so that it is applied to all future outputs, select “Edit” from the top main SPSS Data View menu. Then select “Options” from the drop down menu followed by the tab “Pivot Tables”. Select the “Browse” button from beneath the “Table View” menu, then select the new look which you created.
6. Clicking “Apply”, followed by “OK”, will apply this look to all future tables created during the duration of the SPSS session.
If you ever want to revert back to the default look, follow the previous steps, and select “<System Default>” from the leftmost “TableLook” menu.
Per Wikipedia, “A Markov chain is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state of the attained in the previous event”.
Explained in a less broad manner, a Markov chain could be described as a way of assessing probabilistic systems by assessing fluidity as it applies to both a single variable, and the other variables contained within a system.
For example, in the case of weather systems, a day which is cloudy may subsequently be followed by a day which is also cloudy, a day without clouds, or a rainy day. However, the probability of each subsequent event will undoubtedly be impacted by the composition of the current state.
Another example of the applied methodology is assessment of market share. If company A offers a product which potentially retains 60% of its current consumers annually, but also has the potential to lose 40% of that consumer base to company B on an annual basis, and company B potentially retains 80% of its current annually, but also has the potential to lose 20% of that consumer base to company A, what is the impact of the phenomenon described on an annual basis?
Let’s explore both examples:
First, we’ll create a model which can predict weather.
We’ll assume that the following probabilities appropriately describe the autumn forecasts for weather in Winnipeg.
Cloudy Clear Snowy Rainy
Cloudy 33% 17% 25% 25%
Clear 25% 50% 12% 13%
Snowy 19% 15% 33% 33%
Rainy 20% 20% 10% 50%
To further understand this probability matrix, assume that currently the day’s forecast in Winnipeg is “Cloudy”. This would typically indicate that the following day would have weather which is either “Cloudy” (33%), “Clear” (17%), “Snowy” (25%), or “Rainy” (25%).
Now, we’ll run the information through the R-Studio platform:
EXAMPLE A – Weather Model
# With the libraries ‘markovchain’ and ‘diagram’ downloaded and enabled #
Weather A 4 - dimensional discrete Markov Chain defined by the following states: Cloudy, Clear, Snowy, Rainy The transition matrix (by rows) is defined as follows: Cloudy Clear Snowy Rainy Cloudy 0.33 0.17 0.25 0.25 Clear 0.25 0.50 0.12 0.13 Snowy 0.19 0.15 0.33 0.33 Rainy 0.20 0.20 0.10 0.50
# Illustrate the Matrix Transitions #
plotmat(trans_mat,pos = NULL,
lwd = 1, box.lwd = 2,
cex.txt = 0.8,
box.size = 0.1,
box.type = "circle",
box.prop = 0.5,
box.col = "light yellow",
arr.length=.1,
arr.width=.1,
self.cex = .4,
self.shifty = -.01,
self.shiftx = .13,
main = "")
This produces the output graphic:
(As it pertains to the graphic- something important to note is the direction of the arrows. The arrow direction in the graphic is inverted. Therefore, I would only use the graphic as an auxiliary for personal reference.)
# We will assume that the current forecast is cloudy by creating the vector below #
Current_state<-c(1, 0, 0, 0)
# Now we will utilize the following code to predict the weather for tomorrow #
steps<-1
finalState<-Current_state*disc_trans^steps
finalState
# Console Output #
Cloudy Clear Snowy Rainy [1,] 0.33 0.17 0.25 0.25
This output indicates that tomorrow will have a 33% chance of being cloudy, a 17% chance of being clear, a 25% chance of being snowy, and a 25% chance of being rainy.
# Let’s predict the weather for the following day #
With this information, we can assume that generally there is a 24% chance of rain, a 26% chance of the day being clear, an 18% of the day being snowy, and a 31% chance of the day being rainy.
It would be helpful if the rounded figures summed to 1. But I think that you probably understand the example regardless.
EXAMPLE A – Market Share
Let’s re-visit our market share example:
Company A offers a product which potentially retains 60% of its current consumers annually, but also has the potential to lose 40% of that consumer base to company B on an annual basis, and company B potentially retains 80% of its current annually, but also has the potential to lose 20% of that consumer base to company A, what is the impact of the phenomenon described on an annual basis?
Let’s make a few assumptions.
First, we will assume that the projection given above is accurate.
Next, we’ll assume that the total customer base as it pertains to the product is 60,000,000.
Finally, we’ll assume that the Company A possesses 20% of this market, and Company B possesses 80% of this market. 12,000,000 individuals and 48,000,000 respectively. # With the libraries ‘markovchain’ and ‘diagram’ downloaded and enabled #
# Console Output # Market Share A 2 - dimensional discrete Markov Chain defined by the following states: Company A, Company B
The transition matrix (by rows) is defined as follows: Company A Company B Company A 0.6 0.4 Company B 0.8 0.2
# Illustrate the Matrix Transitions #
plotmat(trans_mat,pos = NULL,
lwd = 1, box.lwd = 2,
cex.txt = 0.8,
box.size = 0.1,
box.type = "circle",
box.prop = 0.5,
box.col = "light yellow",
arr.length=.1,
arr.width=.1,
self.cex = .4,
self.shifty = -.01,
self.shiftx = .13,
main = "")
This produces the output graphic:
(Again, as it pertains to the graphic- something important to note is the direction of the arrows. The arrow direction in the graphic is inverted. Therefore, I would only use the graphic as an auxiliary for personal reference.)
# We will assume that the market share is as follows #
# This reflects the information provided in the example description above #
Current_state<- c(0.20,0.80)
# Now we will utilize the following code to predict the market share for the next year #
steps<-1
finalState<-Current_state*disc_trans^steps
finalState
# Console Output #
Company A Company B [1,] 0.76 0.24
As illustrated, one year out, Company A now controls 76% of the market share (45,600,000)*, and Company B controls 24% of the market share (14,400,000).
* Assuming that original market share does not increase or decline in overall individuals. The calculation for the figures is: 60,000,000 * .76 and 60,000,000 * .24.
Similar to our previous example, we can also project the current trend for multiple consecutive time periods.
# The following code to predicts the market share for the following two years #
steps<-2
finalState<-Current_state*disc_trans^steps
finalState
# Console Output #
Company A Company B [1,] 0.648 0.352
Steady state in the case of this example, will predict the potential equilibrium which will be reached if the trends continue ad infinitum.
# Steady state Matrix #
steadyStates(disc_trans)
# Console Output #
Company A Company B [1,] 0.6666667 0.3333333
Company A in this scenario now controls approximately 66.66% of the market share, and Company B controls 33.33% of the market share.
In prior articles, I explained the various test of correlation which are available within the R programming language. One of those methods which was described but is rarely utilized outside of the textbook, is the Distance Correlation T-Test methodology.
In this entry, I will briefly explain when it is appropriate to utilize the distance correlation, and how to appropriate apply the methodology within the R framework.
Now I must begin by stating that what I am about to describe is uncommon, and should only be utilized in situations which absolutely warrant application.
The distance correlation as described within the context of this blog is:
Distance Correlation – A method which tests model variables for correlation through the utilization of a Euclidean distance formula.
So when would I apply the Distance Correlation T-Test? To answer this question, only in situations in which other correlation methods are inapplicable. In the case which I am about to demonstrate, an example of the inapplicability of other methods would be situations in which one variable is continuous, and the other is categorical.
Example:
(This example requires that the R package: “energy”, be downloaded and enabled.)
There was a not significant difference in GROUP X (M = 4.80, SD = 3.79), as compared to GROUP Y (M = 79, SD = 14.35), t(34) = -0.11, p = .55.
However, you may be wondering, what is the difference between the Distance Correlation T-Test, the Distance Correlation Method, and the Pearson Test of Correlation?
Distance Correlation T-Test – Utilized to test for significance in situations in which one variable is continuous, and the other is categorical. This method can also be utilized in other situations, however, if both variables are continuous, then the Pearson Test of Correlation is most appropriate.
Distance Correlation Method – Utilized to test for correlation between two variables when assessed through the application of the Euclidean Distance Formula. This model output value is similar to coefficient of determination, in that, it can range from 0 (no correlation), to 1 (perfect correlation).
The Pearson Test of Correlation – Utilized to determine if values are correlated. This method should typically be utilized above all other tests of correlation. However, it is only appropriate to utilize this method when both variables are continuous.
There are many model types, methods and techniques demonstrated on this website. In this entry, I will categorize each of the aforementioned concepts, and provide a brief description as it pertains to the scenario which would warrant appropriate utilization.
(Tests of Normality)
Q-Q Plot – A graph which is utilized to assess data for normality.
P-P Plot – A graph which is utilized to assess data for normality.
Shapiro-Wilk Normality Test – A test which is utilized to test data for normality.
(Tests Related to Parametric Model Variable Correlation)
Variance Influence Factor– A method which tests model variables for correlation.
(Pearson) Coefficient of Correlation – A method which tests variables for correlation.
Partial Correlation - A method which is utilized to measure the correlation between two variables, while also controlling for a third variable.
Distance Correlation – A method which tests model variables for correlation through the utilization of a Euclidean distance formula.
Canonical Correlation – A method which assesses model variables for correlation through the combination of model variables into independent groups.
(Tests Related to Non-Parametric Model Variable Correlation) Spearman’s Rank Correlation- A non-parametric alternative to the Pearson correlation. This method is utilized in circumstances when either data samples are non-linear, or the data type contained within those samples are ordinal. An example of ordinal data – “survey response data which asked the respondent to rank a particular item on a scale of 1-10”.
Kendall Rank Correlation Coefficient - Like Spearman’s rho, Kendall’s Tau is also utilized in circumstances when either data samples are non-linear, or the data type contained within the samples is ordinal.
(Tests of Significance Amongst Groups)
One Sample T-Test - This test is utilized to compare a sample mean to a specific value, it is used when the dependent variable is measured at the interval or ratio level.
Two Sample T-Test - This test functions in the same manner as the above test. However, in the case of this model, data is randomly sampled from different sets of items from two separate control groups.
The Welch Two Sample T-Test - This test functions in the same manner as the above test. The only difference being, this method is utilized if data is randomly sampled from different sets of items from two separate control groups of uneven size.
Paired T-Test– Similar in composition to the Two Sample T-Test, this test is utilized if you are sampling the same set twice, once for each variable.
(Analysis of Variance “ANOVA”) Analysis of Variance– Also known as ANOVA, this method is utilized to test for significance across the variances of multiple sample groups. In many ways, this test is similar to a t-test, however, ANOVA allows for multiple group comparison.
One Way Analysis of Variance (ANOVA)– An ANOVA model containing a single independent variable.
Two Way Analysis of Variance (ANOVA) - An ANOVA model containing multiple independent variables.
Repeated-Measures Analysis of Variance (ANOVA) – An ANOVA model containing a single independent variable measured multiple times.
(Exotic Analysis of Variance “ANOVA” Variants)
Analysis of Covariance (ANCOVA)– An ANOVA model which also factors for a covariate value which may impact the system as a whole.
Multivariate of Covariance (MANCOVA) – An ANOVA model containing multiple dependent variables. Also factors for a covariate value which may impact the system as a whole.
Friedman Test (One Way Analysis of Variance) – The nonparametric alternative to a One Way ANOVA test.
Wilcox Signed Rank Test (One Sample T-Test, Paired T-Test) – The nonparametric alternative to the One Sample T-Test, and the Paired T-Test.
Mann-Whitney U Test (Two Sample T-Test) – A nonparametric alternative to the One Way ANOVA test.
(Tests of Significance Amongst Groups)
Chi-Square – A test which measures categorical significance as it pertains to a binary outcome variable.
McNemar's Test– A test which measures categorical significance, limited to two initial categories, and two categorical outcomes. This test is typically utilized for drug trials.
(Metric to Assess Rate of Agreement Amongst Two Entitles)
Cohen’s Kappa– A test which measures the rate of agreement amongst two entities.
(Tests of Significance Amongst Groups Comprised of Survey Questions)
Cronbach’s Alpha - Cronbach’s Alpha is primarily utilized to measure the inter-relatedness of response data collected from sociological surveys. Specifically, the potential differentiation of response information related to certain interrelated categorical survey questions.
(Tests Pertaining to Stationarity and Random Walks)
Dicky-Fuller Test – A methodology of analysis utilized to test data for stationarity.
Phillips-Perron Unit Root Test – A methodology utilized to test data for random walk potential.
(Comparison of Outcome Variables)
Two Step Cluster– A method which assesses model outcome variables through the utilization of a clustering technique.
K-Means - A method which assesses model outcome variables through the utilization of a clustering technique.
Hierarchical Cluster - A method which assesses model outcome variables through the utilization of a hierarchal technique.
K-Nearest Neighbor – A method which compares similarity of outcome variables as determined by the values of the model’s independent variables.
(Reduction of Independent Variables through Variable Synthesis)
Dimension Reduction – A method which creates new variables with values that are determined by the original values of the independent model variables.
(Impact Assessment)
TURF Analysis – A method of analysis typically utilized for product and design studies. This technique assesses the most effective way to reach a sample target demographic.
(Survival Analysis)
Survival Analysis - A statistical methodology which measures the probability of an event occurring within a group over a period of time.
(Sample Distribution Tests)
The Wald Wolfowitz Test- A method for analyzing a single data set in order to determine whether the elements within the data set were sampled independently.
The Wald Wolfowitz Test (2-Sample) - A method for analyzing two separate sets of data in order to determine whether they originate from similar distributions.
The Kolmogorov-Smirnov Test - A method for analyzing a single data set in order to determine whether the data was sampled from a normally distributed population.
The Kolmogorov-Smirnov Test (2-Sample) - A method for analyzing two separate sets of data in order to determine whether they originate from similar distributions.
(Outcome Models – Conditions for Utilization)
Linear Regression – Continuous outcome variable. Continuous independent variable(s).
General Linear Mixed Models – Continuous outcome variable. Any type of independent variable(s).
In today’s article, we will discuss the standard methodology which is utilized to report statistical findings. In previous examples featured on this website, model outputs were explained in a more simplistic manner in order to decrease the level of complexity related to such. However, if the purpose of the overall research endeavor is to produce results for publication, then the APA format should be applied to whatever experimental findings are generated from the application of methodologies.
“APA” is an abbreviation for The American Psychological Association. Regardless of the type of research that is being conducted, the formatting standards maintained by the APA as it applies to statistical research, should always be utilized when presenting data in a professional manner.
Details
All figures which contain decimal values should be rounded to the nearest hundredth. Ex. .105 = .11. Reporting p-values being the exception to this rule. P-values should, in most cases, be reported in a format which contains two decimals. The exception occurring when a greater amount of specificity is required to illustrate the details of the findings.
Another rule to keep in mind pertains to leading zeroes. A leading zero prior to a decimal place is only required if the represented figure has the potential to exceed “1”. If the value cannot exceed “1”, then a leading zero is un-necessary.
Below are examples which demonstrate the most common application of the APA format.
Chi-Square Template:
A chi-square test of independence was performed to examine the relation between CATEGORY and OUTCOME. The relation between these variables was found to be significant at the p < .05 level, χ2 (DEGREES OF FREEDOM, N = SAMPLE SIZE) = X-Squared Value, p = p - value.
- OR -
A chi-square test of independence was performed to examine the relation between CATEGORY and OUTCOME. The relation between these variables was not found to be significant at the p < .05 level, χ2 (DEGREES OF FREEDOM, N = SAMPLE SIZE) = X-Squared Value, p = p - value.
Example:
While working as a statistician at a local university, you are tasked to evaluate, based on survey data, the level of job satisfaction that each member of the staff currently has for their occupational role (Assume a 95% Confidence Interval).
The data that you gather from the surveys is as follows:
A chi-square test of independence was performed to examine the relation between occupational role and job satisfaction. The relation between these variables was found to be significant at the p < .05 level, χ2 (3, N = 330) = 18.56, p < .001.
Tukey HSD
Template:
Post hoc comparisons using the Tukey HSD test indicated that the mean score for the CONDITION A (M = Mean1, SD = Standard Deviation1) was significantly different than CONDITION B (M = Mean2, SD = Standard Deviation2), p = p-value.
Analysis of Variance (ANOVA) (One Way)
Template:
There was a significant effect of the CATEGORY on the OUTCOME for SCENARIO at the p <. 05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(1), Degrees of Freedom(2)) = F Value, p = p - value).
- OR -
There was not a significant effect of the CATEGORY on the OUTCOME for SCENARIO at the p <. 05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(1), Degrees of Freedom(2)) = F Value, p = p - value).
Example:
A chef wants to test if patrons prefer a soup which he prepares based on salt content. He prepares a limited experiment in which he creates three types of soup: soup with a low amount of salt, soup with a high amount of salt, and soup with a medium amount of salt. He then servers this soup to his customers and asks them to rate their satisfaction on a scale from 1-8.
Low Salt Soup it rated: 4, 1, 8
Medium Salt Soup is rated: 4, 5, 3, 5
High Salt Soup is rated: 3, 2, 5
(Assume a 95% Confidence Interval) # Code #
satisfaction <- c(4, 1, 8, 4, 5, 3, 5, 3, 2, 5)
salt <- c(rep("low",3), rep("med",4), rep("high",3))
salttest <- data.frame(satisfaction, salt)
results <- aov(satisfaction~salt, data=salttest)
summary(results)
# Console Output #
Df Sum Sq Mean Sq F value Pr(>F)
salt 2 1.92 0.958 0.209 0.816
Residuals 7 32.08 4.583
APA Format:
There not was a significant effect of the level of salt content on patron satisfaction at the p<.05 level for the three conditions (F(2, 7) = 0.21, p = 0.82).
(Two Way)
Template:
Hypothesis 1:
There was a significant effect of the CATEGORY on the OUTCOME for SCENARIO at the p <. 05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(1), Degrees of Freedom(2)) = F Value, p = p - value).
- OR -
There was not a significant effect of the CATEGORY on the OUTCOME for SCENARIO at the p <. 05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(1), Degrees of Freedom(2)) = F Value, p = p - value).
Hypothesis 2:
There was a significant effect of the CATEGORY2 on the OUTCOME for SCENARIO at the p <. 05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(2), Degrees of Freedom(4)) = F Value, p = p - value).
- OR -
There was not a significant effect of the CATEGORY2 on the OUTCOME for SCENARIO at the p <. 05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(2), Degrees of Freedom(4)) = F Value, p = p - value).
Hypothesis 3:
There was a statistically significant interaction effect of the CATEGORY1 on the CATEGORY2 at the p < .05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(3), Degrees of Freedom(4)) = F Value, p = p - value).
- OR -
There was not a statistically significant interaction effect of the CATEGORY1 on the CATEGORY2 at the p < .05 level for the NUMBER OF CONDITIONS (F(Degrees of Freedom(3), Degrees of Freedom(4)) = F Value, p = p - value).
Example:
Researchers want to test study habits within two schools as they pertain to student life satisfaction. The researchers also believe that the school that each group of students is attending may also have an impact on study habits. Students from each school are assigned study material which in sum, totals to 1 hour, 2 hours, and 3 hours on a daily basis. Measured is the satisfaction of each student group on a scale from 1-10 after a 1 month duration.
(Assume a 95% Confidence Interval)
School A:
1 Hour of Study Time: 7, 2, 10, 2, 2
2 Hours of Study Time: 9, 10, 3, 10, 8
3 Hours of Study Time: 3, 6, 4, 7, 1
Df Sum Sq Mean Sq F value Pr(>F) studytime 2 62.6 31.300 3.809 0.0366 * school 1 2.7 2.700 0.329 0.5718 studytime:school 2 7.8 3.900 0.475 0.6278 Residuals 24 197.2 8.217 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
APA Format:
There was a significant effect as it pertains to study time impacting student stress levels at the p < .05 level for the three conditions (F(2, 24) = 3.81, p = .04).
There was not a significant effect as it relates to the school attended impacting student stress levels at the p < .05 level for the two conditions (F(1, 24) = 0.329, p > .05).
There was not a statistically significant interaction effect of the school variable on the study time variable at the p < .05 level (F(2, 24) = 0.475, p > .05).
TukeyHSD(results) > TukeyHSD(results) Tukey multiple comparisons of means 95% family-wise confidence level
Post hoc comparisons using the Tukey HSD test indicated that at the p < .05 level, the mean score for the level of stress exhibited by students who studied for Two Hours (M = 7.20, SD = 2.62), was significantly different as compared to the scores of the students who studied for Three Hours (M = 3.70, SD = 2.00), p = .03.
(Repeated Measures)
Template:
Example:
Researchers want to test the impact of reading existential philosophy on a group of 8 individuals. They measure the happiness of the participants three times, once prior to reading, once after reading the materials for one week, and once after reading the materials for two weeks. We will assume an alpha of .05.
Before Reading = 1, 8, 2, 4, 4, 10, 2, 9
After Reading = 4, 2, 5, 4, 3, 4, 2, 1
After Reading (wk. 2) = 5, 10, 1, 1, 4, 6, 1, 8 library(lme4) # You will need to install and enable this package # library(nlme) # You will also need to install and enable this package #
There was not a significant effect of the health assessment on the survey questions related to stroke concern at the p < .05 level for the five conditions (F(1, 14) = 1.05, p > .05).
Student’s T-Test
(One Sample T-Test) Template:
(Right Tailed)
There was a significant increase in the GROUP A (M = Mean of GROUP A, SD = Standard Deviation of GROUP A), as compared to the historically assumed mean (M = Historic Mean Value); t(Degrees of Freedom) = t-value, p = p-value.
- OR -
There was not a significant increase in the GROUP A (M = Mean of GROUP A, SD = Standard Deviation of GROUP A), as compared to the historically assumed mean (M = Historic Mean Value); t(Degrees of Freedom) = t-value, p = p-value. Example:
A factory employee believes that the cakes produced within his factory are being manufactured with excess amounts of corn syrup, thus altering the taste. 10 cakes were sampled from the most recent batch and tested for corn syrup composition. Typically, each cake should comprise of 20% corn syrup. Utilizing a 95 % confidence interval, can we assume that the new batch of cakes contain more than a 20% proportion of corn syrup?
t.test(N, alternative = "greater", mu = .2, conf.level = 0.95) # " alternative = " Specifies the type of test that R will perform. "greater" indicates a right tailed test. "left" indicates a left tailed test."two.sided" indicates a two tailed test. # One Sample t-test
data: N t = 3.6713, df = 9, p-value = 0.002572 alternative hypothesis: true mean is greater than 0.2 95 percent confidence interval: 0.244562 Inf sample estimates: mean of x 0.289
A one sample t-test was conducted to compare the level of corn syrup in the current sample batch of cakes, to the assumed historical level of corn syrup contained within previously manufactured cakes.
There was a significant increase in the amount of corn syrup in the recent batch of cakes (M = .29, SD = .08), as compared to the historically assumed mean (M =.20); t(9) = 3.67, p = .003.
(Two Sample T-Test)
Template:
(Two Tailed) There was a significant difference in the GROUP A (M = Mean of GROUP A, SD = Standard Deviation of GROUP A), as compared to the GROUP B (M = Mean of GROUP B, SD = Standard Deviation of GROUP B), t(Degrees of Freedom) = t-value, p = p-value.
-OR-
There was not a significant difference in the GROUP A (M = Mean of GROUP A, SD = Standard Deviation of GROUP A), as compared to the GROUP B (M = Mean of GROUP B, SD = Standard Deviation of GROUP B), t(Degrees of Freedom) = t-value, p = p-value.
A scientist creates a chemical which he believes changes the temperature of water. He applies this chemical to water and takes the following measurements:
70, 74, 76, 72, 75, 74, 71, 71
He then measures temperature in samples which the chemical was not applied.
74, 75, 73, 76, 74, 77, 78, 75
Can the scientist conclude, with a 95% confidence interval, that his chemical is in some way altering the temperature of the water?
data: N2 and N1 t = 2.4558, df = 14, p-value = 0.02773 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: 0.3007929 4.4492071 sample estimates: mean of x mean of y 75.250 72.875
A two sample t-test was conducted to compare the temperature of water prior to the application of the chemical, to the temperature of water subsequent to the application of the chemical
There was a significant difference in the temperature of water prior to the application of the chemical (M = 72.88, SD = 2.17), as compared to the temperature of the water subsequent to the application of the chemical (M = 75.25, SD = 1.67); t(14) = 2.46, p = .03.
(Paired T-Test)
Template:
(Right Tailed)
There was a significant increase in the GROUP A (M = Mean of GROUP A, SD = Standard Deviation of GROUP A), as compared to the GROUP B (M = Mean of GROUP B, SD = Standard Deviation of GROUP B), t(Degrees of Freedom) = t-value, p = p-value.
- OR -
There was not a significant increase in the GROUP A (M = Mean of GROUP A, SD = Standard Deviation of GROUP A), as compared to the GROUP B (M = Mean of GROUP B, SD = Standard Deviation of GROUP B), t(Degrees of Freedom) = t-value, p = p-value.
Example:
A watch manufacturer believes that by changing to a new battery supplier, that the watches that are shipped which include an initial battery, will maintain longer lifespan. To test this theory, twelve watches are tested for duration of lifespan with the original battery.
The same twelve watches are then re-rested for duration with the new battery.
Can the watch manufacturer conclude, that the new battery increases the duration of lifespan for the manufactured watches? (We will assume an alpha value of .05).
data: N2 and N1 t = 2.4581, df = 11, p-value = 0.01589 alternative hypothesis: true difference in means is greater than 0 95 percent confidence interval: 12.32551 Inf sample estimates: mean of the differences 45.75 mean(N1) sd(N1)
A paired t-test was conducted to the lifespan duration of watches which contained the new battery, to the lifespan of watches which contained the initial battery.
There was a significant increase in the lifespan duration of watches which contained the new battery (M = 325.33, SD =56.85), as compared to the lifespan of watches which contained the initial battery (M = 371.08, SD = 51.23); t(11) = 2.46, p = .02.
Residuals: Min 1Q Median 3Q Max -6.4016 -5.0054 -1.7536 0.8713 14.0886
Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 47.1434 12.0381 3.916 0.00578 ** x 0.7808 0.3316 2.355 0.05073 . z 0.3990 0.4804 0.831 0.43363 --- Residual standard error: 7.896 on 7 degrees of freedom Multiple R-squared: 0.5249, Adjusted R-squared: 0.3891 F-statistic: 3.866 on 2 and 7 DF, p-value: 0.07394
APA Format:
A linear regression model was utilized to test if variables “x” and “z” significantly predicted outcomes within the observations of “y” included within the sample data set. The results indicated that while “x” (B = .781, p = .051) is a significant predictor variable, the overall model itself does not possess a worthwhile predictive capacity (r2 = .041). (Non-Standard Regression Model)
A logistic regression model was utilized to test if a model containing the variables “Age”, “Smoking Status”, and “Obesity”, could predict Cancer outcomes as it pertains to the individuals included within the sample data set. The results indicated that the model does not possess a worthwhile predictive capacity (Nagelkerke R-Square = .37).
