Two Step Cluster (SPSS)

Two Step Cluster Analysis is a synthetic methodology that utilizes algorithms to create groupings based on similarities between collections of variables within a single data set. The procedure itself is not a statistical concept. You will not find this method discussed within statistics textbooks. Two Step Cluster Analysis only exists within SPSS. The algorithm that creates the output is proprietary, and therefore, cannot be reverse engineered or re-produced by hand. Therefore, I would recommend that this method only be used sparingly for certain situations. Additionally, model output should always be provided along with SPSS syntax (code) and data.

Example:

To create a two step cluster analysis within SPSS, first choose “Analyze” from the top drop down menu. After this option has been selected, choose “Classify”, and then choose “TwoStep Cluster”.

You will be presented with a menu which presents the following options:

For our example model, we will be creating analytic output which includes the categorical variables “Cat_Var1” and “Cat_Var2”. Additionally, we will also include the continuous variable “Cont_Var1”. For “Distance Measure”, if your model does not contain categorical variables, and if you wish to manually specify the “Number of Clusters” which the model will include for analysis, it is best to change the option “Log-likelihood” to “Euclidean”.

TwoStep Cluster Analysis (Menu Explanation)

This menu is establishing the parameters in which the model will adhere to upon its creation. If “Determine automatically” is selected, the output, which will contain the model itself, will comprise of a full analyzation of the selected variables, which therein will comprise of groupings which the computer algorithm determined most appropriate for the situation.

If “Specify fixed” is selected, the computer will put forth its best efforts to create the amount of groupings specified by the user. This forced number of groupings will be utilized for the creation of the model output. (Reminder: If your model will comprise of only categorical variables, and you would like to specify the number of groupings, it is best to change “Distance Measure” to “Euclidean”.)

Example (cont.)

If we were to continue with our example and select “Output” from the above menu, we would be presented with the following:

Let’s select “Cont_Var2” from our “Variables” list. This will move the selection to the “Evaluation_Fields” menu box. Selecting “Create cluster membership variable” beneath the “Working Data File” header will write output to the data table after the model output is provided.

Clicking on “Continue” from this menu, and “OK” from the prior menu, will provide the following output:

Model Summary (Explanation)

Model Summary

Algorithm – This cell entry is providing the algorithm utilized to create the model.

Inputs – This cell entry is providing the number of inputs utilized to create the model.

Clusters – This cell entry is providing the number of clusters produced by the sorting algorithm.

Cluster Quality

This output is illustrating the overall strength of the model.

(Double clicking on the TwoStep Cluster output provides the following illustration)

What is shown in the above output is a graphical illustration of the clusters which combined, represent the model in its entirety.

Chart (Explanation)

Size of Smallest Cluster – This is number of entries from which the smallest cluster is comprised. To the right of this value is the percentage of the model which the cluster represents.

Size of Largest Cluster – This is number of entries from which the largest cluster is comprised. To the right of this value is the percentage of the model which the cluster represents.

Ratio of Sizes: Largest Cluster to Smallest Cluster – This value is representative of the ratio produced when largest cluster is divided by smallest cluster. The value of this ratio should be no greater than 2.

If you change the “View” in the menu below the graphical output from “Cluster Sizes” to “Predictor Importance”, you will be presented with the following graphic:

If you change the “View” in the menu below the model summary from “Clusters” to “Model Summary”, you will be presented with the following graphic:

Within this table, we are presented with the following:

Cluster – Each cluster segmented by a numerical value.

Label – There is no default label provided. However, if you would like to create a label for cluster “1”, this field enables you to do so.

Description - There is no default description provided. If you would like to create a description for cluster “1”, this field enables you to do so.

Size – The size of each cluster as it relates to the total number of observations contained within the model. Percentage of Total Model (number of observations within cluster).

Inputs

Listed in the order of predictive importance are the variables which make up each cluster. If you hover your mouse above any cell, a box will appear which contains a key pertaining to what is represented within the cell.

If a variable is categorical, its most frequent category is listed along with the frequency of its occurrence within the group.

If a variable is continuous, its mean value is listed instead.

You may recall that at the beginning of this exercise that we selected “Cont_Var2” for our “Evaluation Field”. These next steps will demonstrate what this accomplished.

On the bottom right side of the menu bar which is displayed beneath the table graphic, there is a button that reads “Display”. Click this button and then select the option “Evaluation Fields”. This should populate “Cont_Var2” within the “Fields” box. Click “OK” after this variable appears.

This adds a bottom row to the chart which contains the previously selected variable.

This variable was not included in the creation of the model, however, its values are displayed as if it were part of the model. This allows for the comparison of non-model variables to the clusters created by the algorithm.

Returning to the initial data set, you will witness an additional column has been created.

This column indicates the cluster that each observational entry adheres to. This data can be utilized to graph the findings of the model, a topic which will be discussed in our next entry.

“Means” (SPSS)

The “Means” function within SPSS generates summary statistic data and allows for the comparison of categorical means within a data set. This feature is similar to the function summary(), which is found within the “R” platform.

I will demonstrate the premise of the function with an example:

The column: “VAR00001” represents categorical data.

The column: “VAR00002” represents numerical data.

To perform the “Means” function, first select “Analyze” from the top menu, then select “Compare Means”. After this selection has been made, click on the option “Means”.

This should bring up the following menu. I have selected “Options” from this interface, and have selected additional options to be considered for analyzation. The “Dependent List” variable will be your numerical variable, and the “Layer 1 of 1” will contain your categorical variable.

Clicking “OK” produces the output:

Case Processing Summary

Included – These two columns contain the count (N) and percentage of the total of the number of sample values that were included in the analysis.

Cases Excluded – These two columns contain the count (N) and percentage of the total of the number of sample values that were excluded from the analysis. Excluded values are those numerical values which did not contain input data.

Included – These two columns contain the count (N) and percentage of the total of the number of sample values that were included, and excluded, within the analysis.

Report

Mean – The mean value of the total values analyzed from this categorical variable.

N – The number of observed values which are included within the category.

Std. Deviation – The standard deviation of the values contained within the category.

Minimum – The minimum value of the observed set of values contained within the category.

Maximum - The maximum value of the observed set of values contained within the category.

Median – The median value of the observed set of values contained within the category.

How to reproduce this analyzation within the “R” platform:

Cat1 <- c(12)

Cat2 <- c(11, 13, 14, 19)

Cat3 <- c(17, 18)

summary(Cat1)

sd(Cat1)

summary(Cat2)

sd(Cat2)

summary(Cat3)

sd(Cat3)

Which produces the output:

> summary(Cat1)
Min. 1st Qu. Median Mean 3rd Qu. Max.
12 12 12 12 12 12
> sd(Cat1)
[1] NA
>
> summary(Cat2)
Min. 1st Qu. Median Mean 3rd Qu. Max.
11.00 12.50 13.50 14.25 15.25 19.00
> sd(Cat2)
[1] 3.40343
>
> summary(Cat3)
Min. 1st Qu. Median Mean 3rd Qu. Max.
17.00 17.25 17.50 17.50 17.75 18.00
> sd(Cat3)
[1] 0.7071068

Summary Independent-Samples T Test (Two Sample T-Test) (SPSS)

In a previous article, I demonstrated within the “R” platform, how to perform a one sample t-test with only summary information. In this article, we will examine how to perform a two sample t-test within SPSS given only summary information.

Using a slightly modified example problem from a prior exercise:

Example:

A scientist creates a chemical which he believes changes the temperature of water. He applies this chemical to water and takes the following measurements:

Summary Data Pertaining to First Set of Measurements

N = 8

SD = 2.167124

Mean = 72.875

He then measures temperature in samples which the chemical was not applied.

Summary Data Pertaining to the Second Set of Measurements

N = 8

SD = 1.669046

Mean = 75.25

Can the scientist conclude, with a 95% confidence interval, that his chemical is in some way altering the temperature of the water?

To perform this analysis, select “Compare Means” from the “Analyze” drop down, then select “Summary Independent-Samples T Test”. 

You should be presented with the menu below. Simply enter the information that was provided in the example problem and click “OK”. You have the option to change the Confidence Level (%), but since we are assuming a 95% confidence level, doing so is un-necessary.

This presents the output:

Typically, unless stated otherwise for academic purposes, you will assume that variances are equal.

From this output we can conclude:

With a p-value of 0.028 (.028 < .05), and a corresponding t-value of - 2.456, we can state that, at a 95% confidence interval, that the scientist's chemical is altering the temperature of the water.

One-Sample T-Test (SPSS)

In further exploring the utilization of the various built in functions of SPSS, today we will be assessing the usage of the One-Sample T-Test.

A One-Sample T-Test measures the significance of a sample data set’s mean against the known, or assumed, mean of a population.

Example:

A high school gym instructor measures how many push-ups each individual student can perform on the school’s intramural day. His results are as follows:

Is the mean of the set, assuming an alpha of .05, significantly different from the national average of push-ups by student (18)?

To calculate this data is SPSS, first choose “Analyze” from the top menu, then choose “Compare Means”, and finally, select “One-Sample T Test”.

Performing the previous tasks should bring up the menu below. “Test Variable(s)” will be the variable set that you wish to analyze, “Options” will allow you to change the confidence interval percentage. Since our alpha is .05, we will leave the “Confidence Interval Percentage” at 95%.

This produces the output:

Our hypothesis test for this scenario is:

H0: µ = x (The sample mean is equal to the population mean)

H1: µ ≠ x (The sample mean is not equal to the population mean)

Since we are looking for general differentiation, our test will be two tailed.

With a p value of .166, we cannot reject the null hypothesis, and therefore, can assume that the sample mean does not significantly differ from the population mean.

Receiver Operating Characteristic (ROC Curve) (SPSS)

What is a Receiver Operating Characteristic, or ROC Curve, as it is more commonly referred to as, and why should you care? The information provided within this article requires a firm understanding of the prior articles pertaining to logistic regression.

The ROC curve was first utilized during World War II for the analysis of radar signals.* Today, the ROC curve is utilized to illustrate, through the use of a diagram, the measurement of positive predictive results against false positive results. If this sounds complicated, don’t be nervous, the concept of the ROC curve is rather synthetic, meaning, that it is not at inherently intuitive. Therefore, I will do my best to explain its interpretation throughout this entry.

This article utilizes the prior the data set example and output.

To create an ROC curve, select “Analyze” from the drop down menu above the data sheet, then select “ROC Curve”.

Once this has been accomplished, you must specify your “Test Variable” and “State Variable” within the menu interface. In our case, the Test Variable will be: “Predicted Probability”, and the State Variable will be: “Cancer”. You must specify the positive value of the “State Variable”, meaning, that you must specify the value of the “State Variable” which identifies a positive result. This value will typically be “1”, which I have entered into the interface example below. For “Display” options, I have selected: “ROC Curve”, “With diagonal reference line”, “Coordinate points of the ROC Curve”.

This produces the output:

“Case Processing Summary” – This output demonstrates the number of actual positive and negative results contained within the original data set.

“ROC Curve” - What you are witnessing in the diagram above, is sensitivity vs. specificity as it pertains to points within the “Coordiantes of the Curve”. Y = Sensitivity and 

X = 1 – Specificity. I am aware that the graphical co-ordinates are plotted backwards on the graphic, however, I wanted to maintain the order in which they were presented within the table. What the diagram seeks to illustrate, is the predictive capacity of our model at various percentage confidence thresholds. Ideally, we would like to see a ROC curve illustration which contains single point in the upper leftmost corner of the graph. The green line illustrates a perfect binomial outcome (50/50 chance). If our blue line passes below the green line, the point in which this occurs, indicates a model cutoff which would provide probability results worse with than random chance. 

The ROC Curve graphic is providing an illustration of the "Coordinates of the Curve".

What is occurring in this table, is the measurement of positive results and false positive results at each percentage output provided by the logistic regression model.

So, to break this down in simpler terms, assuming that we utilized the value on the right: “Positive if Greater Than or Equal to”, as a cutoff in which to deem all cases of that value or higher as positive results (1), “Sensitivity” would then indicate the number of cases which were actually positive and identified as such (when 1 is predicted and the result is actually 1), “1-Specificity” would indicate the number of cases which were identified as positive by the model, but were actually negative (when 1 is predicted by the result is actually 0).

As an example: While identifying true outcomes based on a model specificity of .806, we will correctly identify 25% of positive outcomes. Therefore, 75% of the total positive outcomes will be overlooked. Additionally, 14.3% outcomes will be identified as positive while they are actually negative. That is to say, that this would be the result if we applied our model to our sample data and considered any result with a logit(p) <= .806 as being negative (0), and any result with a logit(p) >= as being positive (1).

With this in mind, it would not be an exaggeration to refer to “Sensitivity” as the number of positive cases identified by the model (as a percentage). Additionally, we could refer to “1-Specificity” as the number of false positive cases identified by the model (as a percentage).

“Area Under the Curve” – Commonly abbreviated as AUC, this value is representative of exactly what the name indicates. AUC, if we were shading in our graphic, would resemble the following:

For this reason, you can probably assume why you would prefer a higher AUC value, as opposed to the alternative. AUC values cannot, for obvious reasons, exceed the value of 1.

I hope that this entry de-mystified the concept of the ROC curve, and hopefully, has provided you with the confidence and information needed to implement its usage into your own work. Until next time, stay tuned, Data Heads!

* https://en.wikipedia.org/wiki/Receiver_operating_characteristic

Correlated Variables (SPSS)

In a prior article, I discussed correlation as it pertained to the “R” platform. In this entry, I will demonstrate how to generate a similar summary within the “SPSS” platform. We will be utilizing the data from the previous exercise for our example.

From the “Analyze” option on the top menu, select “Correlate” and then “Bivariate”.

Select the variable that will be analyzed. There is no need to select any additional options.

This produces the output below.

Typically, Pearson Correlation (“r”) values are interpreted as follows:

No Association: r value of 0

Small Association: r value of .1 to .3 (or) -.1 to .3

Medium Association: r value .3 to .5 (or) -.3 to -.5

Large Association: r value of .5 to 1 (or) -.5 to -1

Perfect Association: r value of 1

A rough rule of thumb is that only variables that possess a large association should be further examined within the context of the model.

That’s all for now, Data Heads! In the next article, we will discuss the ROC Curve and how it pertains to logistic regression.

(R) Logistic Regression Analysis (Non-Binary Categorical Variables) (SPSS)

In a previous article we covered how to analyze data through the utilization of the logistic regression model. In the example that was presented, categorical data was conveniently binary in every instance. In this example, I will demonstrate how to utilize the logistic regression model when categorical data contains multiple categories.

We will again refer to the example data set below. I have added an additional category that specifies the “Race” of each individual surveyed.

With category labels enabled, the data resembles:

Performing the analysis is similar to the prior example, except, in this case, we will be selecting the “Categorical” option. After doing such, we will specify that “Race” is a categorical covariate. 

Moving forward with our analysis, we will receive the following as a portion of our output:

“Race” has been split into 4 separate variables, with “Race” as a single variable, remaining for evaluation as whole. 

Race(1) refers to the “Race” category: “White”.

Race(2) refers to the “Race” category: “African American”.

Race(3) refers to the “Race” category: “Asian”.

Race(4) refers to the “Race” category: “Indian”.

The “Race” category “Native American” is still accounted for within the context of the model. However, its value is that of the constant in addition to all other variables.

In this example case, our equation would resemble:

Logit(p) = -2.13055 + (Age * 0.03335) + (Obese * -0.56859) + (Smoking * 3.02867) + (White * -1.10077) + (African_American * -1.05379) + (Asian * -1.22213) + (Indian * 0.69143)

So, if we wanted to test our model probability for an individual who was:

55 Years of Age

Obese

A Smoker

White

The equation would resemble:

Logit(p) = -2.13055 + (55 * 0.03335) + (1 * -0.56859) + (1 * 3.02867) + (1 * -1.10077) + (0 * -1.05379) + (0 * -1.22213) + (0 * 0.69143)

So our logit(p) value would be: 1.06301

Which equals a positive probability of: 0.7432653

Additionally, if our model was tested for an individual who was:

26 Years of Age

Not Obese

A Smoker

Native American

Our equation would resemble:

Logit(p) = -2.13055 + (26 * 0.03335) + (0 * -0.56859) + (1 * 3.02867) + (0 * -1.10077) + (0 * -1.05379) + (0 * -1.22213) + (0 * 0.69143) 

Logit(p) would equal: 1.76522

Which equals a positive probability of: 0.8538622

You can test this model in R with the following code:

# Model Test Code #

Age <- 0

Obese <- 0

Smoking <- 0

White <- 0

African_American <- 0

Asian <- 0

Indian <- 0

p <- -2.13055 + (Age * 0.03335) + (Obese * -0.56859) + (Smoking * 3.02867) + (White * -1.10077) + (African_American * -1.05379) + (Asian * -1.22213) + (Indian * 0.69143)

plogis(p) 

Here is how you would create the same model through the utilization of the “R” Platform:

# Non-Binary Categorical Variables #

Age <- c(55, 45, 33, 22, 34, 56, 78, 47, 38, 68, 49, 34, 28, 61, 26)

Obese <- c(1,0,0,0,1,1,0,1,1,0,1,1,0,1,0)

Smoking <- c(1,0,0,1,1,1,0,0,1,0,0,1,0,1,1)

Cancer <- c(1,0,0,1,0,1,0,0,1,1,0,1,1,1,0)

White <- c(1,1,1,0,0,0,0,0,0,0,0,0,0,0,0)

African_American <- c(0,0,0,1,1,1,0,0,0,0,0,0,0,0,0)

Asian <- c(0,0,0,0,0,0,1,1,1,0,0,0,0,0,0)

Indian <- c(0,0,0,0,0,0,0,0,0,1,1,1,0,0,0)

Native_American <- c(0,0,0,0,0,0,0,0,0,0,0,0,1,1,1)

CancerModelII <- data.frame(Age, Obese, Smoking, Cancer, White, African_American, Asian, Indian, Native_American )

CancerModelLogII <- glm(Cancer~ Age + Obese + Smoking + White + African_American + Asian + Indian + Native_American, family=binomial)

summary(CancerModelLogII)

# Which produces the output #

Call:
glm(formula = Cancer ~ Age + Obese + Smoking + White + African_American +
Asian + Indian + Native_American, family = binomial)

Deviance Residuals:
      Min       1Q      Median      3Q        Max
-1.9613 -0.7252    0.4240    0.8107   1.7092

Coefficients: (1 not defined because of singularities)
                                             Estimate        Std. Error      z value     Pr(>|z|)
(Intercept)                           -2.13055         2.58207         -0.825       0.409
Age                                      0.03335         0.04641           0.719       0.472
Obese                                 -0.56859         1.60680          -0.354       0.723
Smoking                              3.02867          1.95858          1.546        0.122
White                                 -1.10077          2.35673          -0.467       0.640
African_American             -1.05379          2.18843          -0.482       0.630
Asian                                 -1.22213          2.40838          -0.507       0.612
Indian                                 0.69143          2.51153            0.275       0.783
Native_American                 NA                 NA                  NA             NA

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 20.728 on 14 degrees of freedom
Residual deviance: 15.366 on 7 degrees of freedom
AIC: 31.366

Number of Fisher Scoring iterations: 4

(R) Logistic Regression Analysis (Binary Categorical Variables) (SPSS)

Today we will be discussing an advanced topic, but a useful topic nonetheless, that topic being: Logistic Regression. Before delving into this subject matter, I would advise you, the reader, if you are not already familiar with linear regression, to please review the articles pertaining to this topic.

Logistic regression is a method which is similar to linear regression, however, the logistic regression method is utilized specifically to create models which analyze dependent variables that are binary in nature (True/False, Yes/No, Positive/Negative, 1/0, etc.).

Let’s begin with a simple example, I have previously entered some sample data into SPSS.

As we discussed in a previous article, you should edit the variable labels so that they correspond with the appropriate binary outcomes.

To perform a logistic regression analysis, select "Analyze" from the top drop down menu, then select "Regression". From the next menu, select "Binary Logistic".

The next screen presents us with the options necessary to structure the equation utilized to create the model. The logistic regression model is structured in a way that is similar to the linear regression model. In the case of our example, “Cancer” will be our dependent variable. “Age”, “Obese”, and “Smoking” will be the model’s independent variables.

NOTE: In this example, our categorical variables are binary. Meaning, that they can only represents two values (0,1). If we were working with sample data that contained categorical variables that were not binary, we would have to specify this in the following screen. This menu can be toggled by clicking on the “Categorical” option.

From the options menu, select “Hosmer-Lemeshow goodness-of-fit”. This test will be useful in interpretating the model results.

Selecting “Save” from the menu options, will present you with this interface. Check “Probabilities” and “Group Membership” beneath the “Predicted Values“ menu header. These options being enabled will output probability data directly to our SPSS spreadsheet.

So what does all of the generated output indicate?

For the most part, you can ignore all of the data listed in the screen below:

Block 0

(Ignore Block 0)

Block 1

Starting from Block 1 to perform analysis, please first observe the portion of this output which reads “Omnibus Tests of Model Coefficients”. We will primarily be concerned with the bottom-most line which coincides with the row labeled, "Model".

The rightmost column entry: "Sig.", indicates the significance level of the model. Depending on the pre-determined confidence interval which has been established, this value will determine whether or not the model is significant. "df" represents the test model's degrees of freedom. The "Chi-square" column contains the corresponding test statistic. All three values, in addition to the number of observations within the model, are utilized in tandem when documenting the model's significance.    

“-2 Log Likelihood”, “Cox & Snell R Square”, “Nagelkerke R Square” – All three columns correspond with outputs generated from methodologies which test the strength of the model.

Cox & Snell R Squared – This value can reach a maximum of .75. Therefore, I believe that it is best to consult the “Nagelkerke R Square” value when assessing the overall model.

Nagelkerke R Square – This value can reach a maximum of 1. Thus, it is the measurement that best resembles the equivalent of The Coefficient of Determination. I would recommend referring to this value when considering the strength of the predictive model.

The “Homer and Lameshow Test” – This is measuring for co-linearity amongst the independent variables of the model. Only the significance of this test is valuable. If the significance percentage is lower than .05, a correlation may be present. In such cases,  additional testing should be performed to address such. Since this test utilizes a Chi-Squared distribution, it is sensitive to sample size. The test is considered most accurate when a sample size is greater than 400.

“Contingency Table for Hosmer and Lameshow Test” – This can be ignored.

“Classification Table” – This output is simple to understand but difficult to follow. What is being illustrated by the output is the number of positive and negative cases predicted by the model, and the number of actual cases that occurred within the sample.

For our model, there were 5 predicted non-occurrences of cancer, and 2 occurrence of cancer that were not predicted. Thus, the percentage of correct estimates is 71.4. (5/(5+2))

Also, there were 2 predicted occurrences of cancer, and 6 non predicted occurrence. Thus, the percentage of correct estimates is 75. (6/(6+2))

The total percentage of accurate estimations provided by the model is 73.3. ((6+5)/(6+2+5+2).

“Variables in the Equation”

The first column lists the model variables. “Age”, “Obese”, “Smoking” are dependent variables. “Constant” is a variable which will be included as an aspect of the model.

So, if we were to construct the model as an equation, it would resemble:

Logit(p) = (Age * .030) + (Obese * -.389) + (Smoking * 2.544) – 2.344

S.E - represents the standard error of each variable within the context of the model.

Wald - represents the value of the Wald Test variable. You can ignore this value.

df - is indicating the degrees of freedom for each variable.

Sig -  is indicating the significance of each variable within the context of the model. An insignificant variable value (typically > .05), does not necessarily indicate that the variable itself should be excluded from subsequent model generation.

Exp(B) or “Odds Ratio” – This indicates the value in which the probability of a positive outcome will increase if the corresponding variable is increased by one. This warrants further explanation, as logistic regression models utilize the “Log” function to generate results.

Returning to our original model equation:

Logit(p) = (Age * .030) + (Obese * -.389) + (Smoking * 2.544) – 2.344

Let’s assume that we wish to test our model on a 19 year old, who is not obese, and who smokes. The equation would resemble:

Logit(p) = (19 * .030) + (0 * -.389) + (1 * 2.544) – 2.344

Logit(p) = .77

But Logit(p) does not equal probability. It equals the Logit value of .77.

To generate the associated probability of this value, you can enter the following information into the R console:

plogis(.77)

Which produces the console output:

[1] 0.6835209

Additionally, you could also create the model within “R” by utilizing the code:

a <- 19
b <- 0
c <- 1

p <- (.030 * a) + (-.389 * b) + (2.544 * c) - 2.344

plogis(p)

Which produces the console output:

[1] 0.6835209

In both cases, the output value is the probability value of a positive occurrence. In the case of our scenario, this would equate a cancer diagnosis. The numbers from our model are un-realistic, but this only due to the absolutely un-realistic sample data that I hastily created.

Returning to the odds ratio.

The value given in the column Exp(B) is the exponential value of the value given in column B. What this value is indicates is the probability increase, within the context of the model, if the variable value of the corresponding B variable is increased by one. However, this value must first be transformed.

Assuming the context of the model, if an individual were being assessed for cancer, assuming that he aged a year, the probability of cancer would increase:

(1.031 – 1) * 100

3.1 %

Data Sheet Output

Previously, during this exercise, I asked that you select “Save” from the menu options, and then subsequently select “Probabilities” and “Group Membership”. In doing so, we have informed SPSS to output data directly to our datasheet. The results are listed below.

We are presented with two new variables “PRE_1” and “PGR_1”. “PRE_1” represents the probability (.00 – 1.00) of a positive event occurring when the dependent variable data found in the left most columns is input into the model. “PGR_1” represents the model’s prediction given the dependent variable data (1 or 0).  This data output is needed to create a ROC curve. ROC curves will be covered in a later article.

Model Creation within R

I will now briefly illustrate how to obtain the same results in R.

# Model Creation #

Age <- c(55, 45, 33, 22, 34, 56, 78, 47, 38, 68, 49, 34, 28, 61, 26)

Obese <- c(1,0,0,0,1,1,0,1,1,0,1,1,0,1,0)

Smoking <- c(1,0,0,1,1,1,0,0,1,0,0,1,0,1,1)

Cancer <- c(1,0,0,1,0,1,0,0,1,1,0,1,1,1,0)

CancerModel <- data.frame(Age, Obese, Smoking, Cancer) 

# Analyze the Significance of the Model #

CancerModelLog <- glm(Cancer ~ Age + Obese + Smoking, family=binomial)

CancerModelLog1 <- glm(Cancer ~ 1, family=binomial)

anova(CancerModelLog, CancerModelLog1, test="LRT")

# Console Output #

Analysis of Deviance Table

Model 1: Cancer ~ Age + Obese + Smoking
Model 2: Cancer ~ 1
Resid. Df Resid. Dev Df Deviance Pr(>Chi)
1 11 16.807
2 14 20.728 -3 -3.9209 0.2701

# Summary Creation and Output # 

CancerModelLog <- glm(Cancer~ Age + Obese + Smoking, family=binomial)

summary(CancerModelLog)

# Output #

Call: 

glm(formula = Cancer ~ Age + Obese + Smoking, family = binomial)

Deviance Residuals:
    Min         1Q          Median     3Q        Max
-1.6096     -0.7471     0.5980    0.8260  1.8485

Coefficients:
                      Estimate      Std. Error    z value     Pr(>|z|)
(Intercept)     -2.34431       2.25748        -1.038      0.2991
Age                0.02984       0.04055          0.736      0.4617
Obese            -0.38924      1.39132         -0.280      0.7797
Smoking        2.54387       1.53564          1.657      0.0976 .
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 20.728 on 14 degrees of freedom
Residual deviance: 16.807 on 11 degrees of freedom
AIC: 24.807
Number of Fisher Scoring iterations: 4

# Generate Nagelkerke R Squared #

# Download and Enable Package: "BaylorEdPsych" #

PseudoR2(CancerModelLog)

# Console Output # McFadden   Adj.McFadden    Cox.Snell    Nagelkerke    McKelvey.Zavoina      Effron 0.2328838    -0.2495624         0.2751639    0.3674311    0.3477522               0.3042371 0.8000000 Adj.Count          AIC          Corrected.AIC 0.5714286      23.9005542      27.9005542 # Calculate Exp(B) # # (Intercept) # exp(-2.34431) # Age # exp(0.02984) # Obese # exp(-0.38924) # Smoking # exp(2.54387) # Output # [1] 0.09591336 [1] 1.03029 [1] 0.6775716 [1] 12.72884 # Utilize VIF() and COR() in lieu of The Homer and Lameshow Test #

# Generate Correlation Matrix # 

correlationmatrix <- cor(CancerModel)

# Output #

                              Age             Obese          Smoking       Cancer
Age                 1.00000000     0.1231756  -0.2836428    0.02147097
Obese             0.12317556    1.0000000   0.4642857    0.19642857
Smoking       -0.28364280     0.4642857  1.0000000     0.46428571
Cancer          0.02147097    0.1964286   0.4642857    1.00000000

# Generate Variance Influence Factor #

# Download and Enable Package: "car" #

vif(CancerModelLog)

# Output # 

     Age        Obese    Smoking
1.286936 1.361140 1.654232