Internal Validity in Longitudinal Homeless Research
Establishing Internal Validity in Longitudinal Research with the Homeless
Introduction
When working with the homeless, it is necessary to identify potential factors that may contribute to the process of entering or exiting homelessness. These factors may take the form of demographic information, socio-economic status, and familial support, to name a few (Johnson et al., 1997; Chamberlain & Johnson, 2013). Some of these variables may be described as negative reinforcements, in that they exacerbate a person’s likelihood of experiencing prolonged homelessness (Aubry, Klodawsky, & Coulombe, 2012). These may include substance use, mental illness, arrest history, and absence of support network (Fazel et al., 2008). The temporal relationship between variables of this nature and homelessness is of particular interest to researchers. Determining whether substance use or mental illness precede and predict one’s chances of entering homelessness, or whether these variables have a greater chance of occurring following the manifestation of homelessness, will have implications for the development of interventions.
While substance use and mental health disorders are shown to occur at increased rates among the homeless as compared to the general population (Fazel et al., 2008), teasing out their particular relationship with the onset, life course, or outcome of homelessness may be difficult to do. In their longitudinal investigation of 344 single adults recruited from municipal homeless shelters in the New York City area, McQuistion, Gorroochurn,Hsu, andCaton (2013) sought to measure the constructs of substance use and mental health, among others, to discover what relationship they had with whether or not someone experienced chronic homelessness, recurring homelessness, or successful rehousing over an eighteen-month period. The authors hypothesized that recurrent homelessness would be associated with characteristics that limit or impede a person’s ability to function, and additionally sought to determine if these characteristics may be independently predictive of recurrent homelessness, or if they are associated with other outcomes (McQuistion et al. p. 2, 2013).
Defining the variables
The dependent variable in this report is described as the life course of experienced homelessness. The researchers limited their participants to exclusively include those who were experiencing homelessness for the first time, so as to observe differences in individual characteristics of those who go on to experience recurrent or chronic homelessness and those are rehoused. The authors recruited participants from the municipal shelter system, and relied on retrospective self-report to measure the continued progress of housing status. Interviews were conducted every six months, while brief check-in interviews were conducted monthly, in an effort to reduce recall bias. As the study proceeded, the authors divided participants into one of three categories: (1.) those experiencing recurrent homelessness – one or more further lapses of homelessness following rehousing, (2.) chronic homelessness – the absence of any housing following baseline interview, and (3.) stably housed – the acquisition and successful retaining of fixed permanent dwelling (McQuistion et al. p. 3, 2013). As a dependent variable, life course of homelessness in this study is sufficiently nuanced to include a wide range of possible experiential outcomes over a span of time, but the concept of “homelessness” itself is narrowed by the restraints of the study’s recruitment technique. While drawing their entire recruitment pool from the municipal shelter system of New York City ensured that participants were experiencing true homelessness (McQuistion et al. p. 2, 2013), this definition of homelessness still excludes those who may be sleeping in cars, residing in homeless encampments, occupying public spaces, or otherwise absent from the shelter system. There will be no way to say whether the results obtained in this study would be any different for people who may not utilize shelters upon entering homelessness.
The independent variables in this study are described as “risk variables” (McQuistion et al., p. 3, 2013), demographic characteristics, and personal history information. The authors describe only a few of the instruments that were used in gathering this information. Upon initiating the baseline interviews, participants were screened for criteria of DSM-IV Axis I disorders (including substance abuse disorders). The Structured Clinical Interview for DSM-IV was used for this purpose, for the sake of brevity. The only Axis II diagnosis screened was antisocial personality disorder, because it is the only Axis II disorder in which behavioral history is the primary criteria (McQuistion et al. p. 2, 2013). While this may be convenient, excluding the diagnosis of other personality disorders further limits the generalizability of this data. History of living arrangement, education, income, employment history, criminal justice involvement, history of childhood placement, and current familial support were also obtained (McQuistion et al,. 2013).
“Out-of-home placement” in childhood was defined as residing with a non-relative before the age of 18 (McQuistion et al., p. 3, 2013). Once again, the definition of this construct may be too narrow in scope, as it overlooks those who have had a similar “out-of-home placement” experiences, but have been placed with distant relatives through foster care. Familial disorganization during childhood was assessed by asking a series of questions related to parental substance abuse, parental criminality, family violence, and other similar items. According to the authors, “family disorganization” as a construct had a reliability ? coefficient of .71 (McQuistion et al. p. 3, 2013). Other reliability coefficients for the remaining instruments were not disclosed.
Relationship between variables
Following data collection after eighteen months, cases were divided into the aforementioned three categories of homeless life course (McQuistion et al. p. 3, 2013). The authors then used multinomial logistic regression analysis to investigate the relationship between each of the housing categories and the risk variables, while controlling for demographic characteristics (McQuistion et al. p. 6, 2013). Some noteworthy associations were discovered.
On its own, substance abuse was associated with increased rates of recurrent homelessness when examined in a bivariate analysis (McQuistion et al. p. 8, 2013). Among the risk variables and dependent variables, no isolated variable was statistically significantly associated with housing status outcome following multinomial logistic regression analysis. However, the authors point out that upon combining three factors – (1.) substance abuse within 30 days prior to baseline interview, (2.) history of arrest, and (3.) a diagnosis of antisocial personality disorder – an outcome of recurrent homelessness could be exclusively predicted. These findings suggest that while no single variable may predict the life course of homelessness, a grouping of risk factors may increase the likelihood of one outcome over another. There are associations, particularly between substance use and the homeless life course, but they may not reach critical influence unless they occur in conjunction with other factors. These findings further illustrate the idea that the phenomenon of homelessness is complex, hard to explain, and involves the culmination of many forces (McQuistion et al., 2013).
Discussion – internal validity
The internal validity at issue in this study comes down to establishing the relationship between three statistically significant risk factors and an outcome of recurrent homelessness, specifically whether one causes the other. While the authors took steps to safeguard against the threat of confounding and selection bias by virtue of the potential independent variables they accounted and controlled for, there are still issues with establishing internal validity. Although the three variables that were collectively linked with recurrent homelessness were temporally established as preceding the outcome, there are alternative explanations for this. Arrest history and antisocial personality disorder have historically been closely related (Hodgins, & Cote, 1993; McCabe et al., 2012). That these both occurred together is redundant, and suggests that one variable that could have covered both of these simultaneously had to be divided to produce the appearance of a significant association. Furthermore, the authors describe a substance use disorder within the past thirty days of the baseline interview as being the third predictive variable for recurrent homelessness. That reported substance abuse was present prior to the baseline interview – and subsequently the first of many recurrent homeless episodes – throws doubt on the temporal assumption of one variable causing the other. Participants could have forseen their entry into homelessness as their support networks fell apart, began using a substance to cope, entered homelessness, and continued using. In this case, entry into homelessness may have brought on substance use, rather than the alternative. That there exists this alternative explanation casts doubt on the internal validity of asserting cause-and-effect between this article’s dependent and independent variables.
While this article does contribute to our understanding of the factors associated with recurrent homelessness – and may even suggest a temporal relationship – it is not flawless. Research attempting to identify the possible causes of a complex phenomenon like homelessness will undoubtedly encounter difficulties in doing so. Regardless, it is the collective contributions of these efforts that will continue to inform our knowledge base, and consequently our interventions, with this population.
References
Aubry, T., Klodawsky, F., & Coulombe, D. (2012). Comparing the housing trajectories of
different classes within a diverse homeless population. American Journal Of Community
Psychology, 49(1-2), 142-155.
Chamberlain, C., & Johnson, G. (2013). Pathways into adult homelessness. Journal Of
Sociology, 49(1), 60-77.
Fazel, S., Khosla, V., Doll, H., & Geddes, J. (2008). The prevalence of mental disorders among
the homeless in Western countries: Systematic review and meta-regression analysis.
PLoS Medicine 5(12), 0001–0012.
Hodgins, S., & Cote, G. (1993). Major mental disorder and antisocial personality disorder: A
criminal combination. Bulletin Of The American Academy Of Psychiatry & The Law,
21(2), 155-160.
Johnson, T. P., Freels, S. A., Parsons, J. A., & Vangeest, J. B. (1997). Substance Abuse and
homelessness: Social selection or adaptation. Addiction, 92, 437–445.
McCabe, P. J., Christopher, P. P., Druhn, N., Roy-Bujnowski, K. M., Grudzinskas, A. r., &
Fisher, W. H. (2012). Arrest types and co-occurring disorders in persons with
schizophrenia or related psychoses. The Journal Of Behavioral Health Services &
Research, 39(3), 271-284.
McQuistion, H. L., Gorroochurn, P., Hsu, E., & Caton, C. M. (2013). Risk factors associated
with recurrent homelessness after a first homeless episode. Community Mental Health Journal, doi:10.1007/s10597-013-9608-4
1
