An Evaluation of SOAR: Implementation and Outcomes of an Effort to Improve Access to SSI and SSDI

Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI), administered by the Social Security Administration (SSA), provide financial assistance, access to health care, and employment assistance for eligible individuals with qualifying disabilities. A substantial portion of people who are homeless or at risk of homelessness have impairments that would qualify them for SSI or SSDI.

Research suggests that at least 46% of individuals who are homeless or at risk of homelessness have one or more chronic general medical conditions (1–3), and as many as 67% have received a psychiatric diagnosis during their lifetimes (4–6). Yet accessing SSI or SSDI is challenging for this population. Their precarious living situations, the combination of disabilities they face (which often include mental health or substance abuse problems), and their tendency to lack social and familial supports make it difficult for them to file a complete and high-quality application. In addition, SSA staff and disability examiners charged with assessing an applicant’s medical condition may have trouble contacting applicants without a stable address and phone number, making it difficult to collect the information necessary to determine eligibility. They may also have trouble obtaining details of the applicant’s medical history, which makes it difficult to develop evidence to support the applicant’s claim.

The SSI/SSDI Outreach, Access and Recovery (SOAR) project, funded by the Substance Abuse and Mental Health Services Administration (SAMHSA), aims to improve access to SSI and SSDI for people who are homeless or at risk of becoming homeless, with a particular focus on people with mental illness. SAMHSA’s SOAR Technical Assistance (TA) Center provides states and local communities with free TA to help them create systems-level change by promoting collaboration among agencies that serve this population and to train professionals who work with the target population to provide assistance throughout the SSI and SSDI application process.

The SOAR TA Center uses SAMHSA’s Stepping Stones to Recovery curriculum to train service providers in assisting people through the SSI and SSDI application process. The curriculum emphasizes ten critical components for improving the quality of SSI and SSDI applications. First, it emphasizes the need to conduct extensive outreach to establish rapport with potential applicants, track their whereabouts, and engage them on an ongoing basis. Other components include becoming an applicant’s authorized representative (which allows the practitioner to provide and receive information on the applicant’s behalf), collecting medical evidence from treatment providers to submit with the application (collaborating with physicians and psychologists for assessments, if necessary), writing and submitting a report that summarizes how the impairment affects the applicant’s daily functioning, and obtaining a quality review of the application from a SOAR expert (such as an agency supervisor or director, an in-state SOAR trainer, or the SOAR TA Center) before submission. Together, these practices are intended to avoid the need for an applicant’s personal treatment provider or an SSA-contracted provider to conduct assessments of the applicant’s general medical or mental health (consultative exams). Disability examiners request a consultative exam if they do not have enough evidence to determine whether an applicant has a qualifying disability.

Until recently, communities that wished to implement SOAR were required to send one or more professionals to an in-person train-the-trainer program, support these professionals in training others to provide application assistance, and facilitate relationships among various entities that support the application process. A key short-term goal of these implementation efforts, for which communities received no direct funding, was to increase the SSI and SSDI application approval rate. The purpose of this study was to examine how SOAR TA has influenced community efforts to implement SOAR and the outcomes of these efforts on approval rates for SSI and SSDI applications. In 2014, the SOAR TA Center replaced the train-the-trainer model with online training to reduce the implementation costs for both the TA Center and communities.

Methods

The study relied on four key sources of data. First, data on SOAR processes were collected from stakeholder interviews (in person and by telephone) and researcher observations in the most active SOAR community in each of 13 states that began receiving TA in fiscal year (FY) 2010 or 2011. Data collection continued in each community for up to three years after implementation of SOAR. Stakeholders included individuals leading SOAR efforts within the community, SOAR practitioners and their supervisors, SOAR trainers, SSA field office staff, staff at state Disability Determination Services (DDS) offices responsible for determining whether an applicant has a qualifying disability, and health care providers.

Second, stakeholders in these communities completed a brief survey about the nature of their relationships and amount of their communication with other stakeholders before and after their participation in SOAR. The sample consisted of 102 stakeholders across all communities, and the response rate was 90% (N=92). Of those who did not respond to the survey, most were unreachable because they had left their positions.

We calculated network density using survey data by creating binary variables indicating whether each stakeholder in the network had communication at least every month or two with the other entities in the network (0=no, 1=yes). We then divided that number by the number of scores of 1 that were possible at each site—for example, at a site with seven stakeholders, each stakeholder could assign a score of 1 for communication with each of the other stakeholders, for a total possible score of 42. Density may be used as a measure of group cohesion or the degree to which the group is unified or tight-knit. We calculated density twice—once before SOAR implementation and once after.

Third, SOAR practitioners in these communities used a management information system (MIS) developed by the SOAR TA Center to track data on their efforts and the outcomes of applications submitted with their help. The study used data entered during a two- to three-year period after implementation of SOAR. Data collection instruments for each of these three study components were cleared by the Office of Management and Budget.

Finally, SSA provided administrative data from its Structured Data Repository and Systems of Records for all individuals who filed SSI and SSDI applications nationally in FY 2010 (N=2,438,944). These data, which contain variables identifying SOAR participants, were used to compare application outcomes for SOAR participants and other SSI and SSDI applicants and to conduct fixed-effects logistic regressions (with robust standard errors calculated using the Huber-White sandwich estimator) to assess the relationship between SOAR participation and the SSI and SSDI application decision. The dependent variable was whether the application was approved (0=no, 1=yes), and the dummy variable was whether the application was submitted through the SOAR process (0=no, 1=yes). Control variables were applicant’s age, primary diagnosis, and state of residence and whether the applicant had submitted an application before. Although many other factors may influence the application decision, we used these control variables because they were the only ones available in the administrative data files for analysis.

The SSA data were also used to predict the probability of application approval associated with use of various SOAR practices emphasized in the training. Because the response variable for this analysis was binary (approval or disapproval), it was necessary to specify the regression model in such a fashion that the probability of the outcome was bound between 0 and 1 for each observation in the data. Therefore, although the outcome of a logistic regression is either 0 or 1, the predicted value for each observation was the probability of an outcome being a success (or a failure).

Results

Training

Over the course of the study, 563 individuals in the 13 communities that were the focus of the process analysis were trained in SOAR. Only 75 (13%), however, completed an application by using the SOAR process. Qualitative data suggested that many trained individuals did not complete applications by using the SOAR process because they had no direct interaction with the target population (administrators and others who do not necessarily work directly with people who are homeless or at risk of homelessness often attend training), they did not have time because of competing obligations (a single application can take between 20 and 40 hours to complete), they left their job before having a chance to provide application assistance (the homeless services workforce generally has high staff turnover), or they did not feel prepared to apply their skills in practice.

Trainees who completed applications by using the SOAR process generally complied with the model’s critical components. According to the SOAR MIS, SOAR trainees helped submit 696 applications that received an initial decision. Almost all of those 696 applications used authorized representatives (N=690, 99%) and included copies of applicants’ medical records (N=663, 95%). Five of the 13 communities developed formal agreements with medical providers—such as state and regional hospital systems, community clinics, correctional agencies, and the state veterans administration—to provide medical records in support of SSI or SSDI applications for free and on an expedited basis. Just over 80% of applications (N=565) were submitted with a report summarizing the applicant’s functioning, and 61% (N=427) were submitted after a quality review. Two-thirds (N=457, 66%) did not require a consultative exam.

The three components analyzed with SSA administrative data—an authorized representative, submission of medical evidence with the application, and avoidance of a consultative exam—used alone or in combination, were predictive of higher approval rates for initial applications compared with applications with none of these components (Figure 1). Submission of medical evidence with the application and avoidance of a consultative exam are likely related, however; the more evidence that is submitted with an application, the less likely it is that the disability examiner will need to gather additional evidence with an exam. In addition, the chances of the application’s approval increase as more of these practices are used.

Regardless of participation in SOAR or an applicant’s living situation, the average likelihood that an initial SSI or SSDI application would be approved was 66% if all three practices were used, compared with 26% if none of them were used. Using only one of the practices raised the likelihood of approval to between 29% and 40%; using two increased the likelihood of approval to between 41% and 56%. Data on use of the other seven components were not available in the SSA files obtained for this study.

Collaboration

Communication among stakeholders increased after the introduction of SOAR in all 13 communities, as measured by network density. On average, across all 13 communities, network density increased by one-third, from eight of 30 scores of 1 at baseline (before SOAR training) to 18 of 30 scores of 1 at the end of the study period, with 30 as the average number of possible scores of 1 across the 13 sites. The increase reflected both new lines of communication and increases in preexisting communication. Across the 13 communities, network density increased over time from 12% (from 19 to 24 of 42 possible scores of 1) to 80% (from 6 to 30 of 30 possible scores of 1).

Stakeholders in all 13 communities reported increased communication with SSA field office and DDS staff. A majority of stakeholders in each community found that SSA and DDS were helpful in assisting people who are homeless or at risk of homelessness to access SSI or SSDI benefits (Figure 2). Few respondents indicated that SSA or DDS had not been helpful to them.

Application Outcomes

SOAR practitioners in the 13 states reported in the MIS that 73% (N=506) of the 696 applications that had received a decision were approved. Two-thirds (N=465, 67%) were approved at initial application, and the rest (N=41, 6%) were approved after a reconsideration of the initial decision. The approval rate among all initial applications in the MIS (N=833), including 137 that did not have an initial decision, was 56%. The number of approved applications across the 13 communities ranged from three to 111 (approval rates of 20% to 100%).

Results from analyses of SSA administrative data indicate that SSI and SSDI applications submitted through the SOAR process were approved at much higher rates than other applications. In FY 2010, SOAR participants were approved for SSI or SSDI at the initial application at almost double the rate for all homeless applicants—50% compared with 28%—and at a substantially higher rate than applicants who were not homeless (35%). Relatively more SOAR participants whose initial application was denied went on to receive reconsideration—46% compared with about 40% of all homeless applicants and applicants who were not homeless. In addition, relatively more SOAR participants (26%) were approved after reconsideration compared with approval rates among all SSI or SSDI applicants who were homeless (14%) or not homeless (12%). Altogether, SSI or SSDI applications were approved for 32% of applicants who were homeless and for 38% of applicants who were not homeless.

Logistic regressions indicated that SOAR participants had higher approval rates relative to non-SOAR participants even after the analyses controlled for other individual and state factors that may influence the application decision (Table 1). Among homeless applicants, the odds of approval of the initial application were significantly higher (odds ratio=2.30) for individuals who went through the SOAR process compared with those who did not.

Discussion

Results suggest that when practitioners provide application assistance by using concepts from SOAR, they help clients achieve better application outcomes. However, training alone is not an efficient use of resources, given that most practitioners who were trained in SOAR never applied SOAR in practice. Communities with relatively high levels of success obtaining application approvals each had practitioners dedicated solely to conducting SOAR application assistance. Communities that expected practitioners to take on SOAR duties along with their other job responsibilities struggled more because staff found it difficult to carve out time for the effort amid competing demands and high caseloads, even when benefit access theoretically was an important component of their work. These findings suggest that communities would benefit from more targeted training efforts, from creating full- or part-time SOAR positions within stakeholder agencies, and from maintaining caseloads that are manageable.

The TA Center estimates that one full-time-equivalent practitioner can complete 50 applications per year. Some communities have secured funds for dedicated SOAR staff through SSA’s Interim Assistance Reimbursement program (which reimburses states for general assistance provided to individuals after the SSI protective filing date and before the approval date), through reimbursement from the Centers for Medicare and Medicaid Services for uncompensated medical care provided between 90 days before the SSI or SSDI protective filing date and the approval date, through grants from federal funding streams, or through state general funds.

The organizational networking facilitated by SOAR communities and supported by the SOAR TA Center successfully increased communication among key stakeholders. The biggest payoff was in new collaborations between SOAR providers and SSA field offices and DDS staff. As the entities responsible for processing applications and making determinations, SSA and DDS have large roles to play in SOAR. Ideally, SSA and DDS staff understand the challenges SOAR applicants face, accommodate those difficulties to the extent possible, and develop relationships with SOAR practitioners, who can provide additional information about applicants. These agencies can facilitate their roles—and have done so in many communities—by designating a point of contact to handle SOAR-related issues and applications and by flagging and expediting applications submitted through SOAR.

Among applicants with similar impairments, applications submitted by using the SOAR process were approved at higher rates compared with applications that did not use SOAR, probably because SOAR helps applicants provide more information to SSA and disability examiners. The more information that is included with an application, the less likely it is that disability examiners will have to find additional evidence to support an applicant’s claim. When disability examiners do not have enough evidence to support a claim, they may ask the applicant to undergo a consultative exam. Consultative exams are particularly problematic for individuals who are homeless or at risk of homelessness. Even if disability examiners have contact information to notify the applicant of the appointment and even if the applicant keeps the appointment, the consultative exam provider may not be able to obtain a complete picture of the individual’s condition and functioning in one short session. The need for a consultative exam, as well as problems collecting or remedying other missing or inadequate information in the application, can delay the claim adjudication process or result in a denial.

This study provides the first independent assessment of SOAR’s outcomes and the mechanisms associated with SOAR’s success. The study had several limitations, however. The communities included in the interviews and observations about process, the social network survey, and MIS data collection and analysis may not be representative of SOAR communities nationally, and the outcomes in the MIS data are based on self-reports from SOAR practitioners and were not independently verified. In addition, SSI and SSDI applicants who were homeless and who submitted applications through the SOAR process were not always identifiable in the SSA administrative data, and these data contained limited variables to use as controls in regression analyses.

Despite this study’s contributions and given its limitations, several questions remain that could be addressed in future studies. For instance, how would the results change if all applicants and SOAR participants who are homeless were properly identified in SSA administrative data? Are better short-term application outcomes for SOAR participants associated with positive long-term outcomes (with respect to income, housing, health, and general well-being)? What would an experimental evaluation of SOAR’s effects show? Planned expansion of SOAR training into new communities through the U.S. Department of Housing and Urban Development’s Continuum of Care program or with the U.S. Department of Veterans Affairs and to new populations (such as staff at hospitals, jails, or prisons) may provide a valuable context in which to embed small randomized control trials.

Conclusions

Findings suggest that providing application assistance by using the SOAR model shows substantial promise for helping individuals who are experiencing or who are at risk of homelessness to access SSI or SSDI and that support from the SOAR TA Center promotes systems-level collaboration toward the goal of improving program access. Applications for SSI and SSDI submitted through the SOAR process were approved at much higher rates than applications that were not associated with SOAR. Promulgating training to provide this application assistance through a train-the-trainer model, however, uses resources inefficiently. The online training that the TA Center began offering in 2014 may prove more cost-efficient and may appeal to individuals most likely to use SOAR in practice.