New LSAT Format will Better Accommodate DisabilitiesYashcheshen v University of Saskatchewan, 2019 SKCA 67 (CanLII)
The Law School Admission Test (LSAT) is a standardized test administered by the American-based Law School Admission Council (LSAC), and is one of the most ubiquitous criteria for law school admissions across North America, including at Canadian law schools. Its use is not without controversy or its detractors, and there are some unique challenges that emerge in administering the LSAT in the context of applicants with medical disabilities.
The LSAT is notably not used in numerous law schools overseas. The Fairness Commissioner confirms that the number of internationally trained lawyers has risen from 7% in 2005, to over a third of all candidates in 2018. The Law Society of Ontario's Professional Development & Competence Committee report to Convocation on Sept. 22, 2016, confirmed that about 35% of the 600 internationally trained candidates in Ontario were Canadian-born, and attend these schools, presumably in large part because of the absence of this LSAT requirement.
The December 2017 application by Ryerson University to the Federation of Law Societies of Canada for the new Ryerson law school cited these figures to place in context the number of domestic graduates they intend to start accepting this Fall 2019. Given the context, Ryerson University will also require LSAT scores for applicants, although there will be a considerable emphasis on the skill sets that Ryerson intends to utilize in its new and innovative program.
The history and the pervasiveness of the exam makes it next to impossible to do away with, even in a modern law school deliberately seeking to break the mold.
The LSAT has been around since about 1948, and was intended to correlate with first-year grades in law school. Since that time, numerous changes have been introduced to the exam, including new policies related to disabilities.
In July 2019, LSAC released a new process for administering a writing sample, separate from the main multiple-choice component of the exam. The types of format-related accommodations currently provided include:
- Unified English Braille (UEB) version of the LSAT
- Large print (e.g., 18-point font or higher) test book
- Screen-readable HTML test (including, where applicable, use of screen-reader software (e.g., JAWS))
- Alternate non-Scantron answer sheet
- Marking answer choices in the test book
- Use of line marker
- Use of magnification devices (e.g., magnification reading glasses, handheld video magnifier, closed-circuit television (CCTV), ZoomText)
- Use of adaptive writing instruments (e.g., thick or felt-tip pen/marker, pencil grips)
- User adjustable Multiple font sizes (8, 10, 12, 15, 18 and 27-point font)
- Adjustable line spacing
- Brightness adjustment
- Large text
- High-contrast text
- Magnification gestures
- Color inversion
- Color filters to assist a wide range of issues, including deuteranomaly (reduced sensitivityto green light), protanomaly (reduced sensitivity to red light), and tritanomaly (reduced sensitivity to bluelight), as well as light sensitivity and visual preferences
The LSAT is designed to measure skills that are considered essential for success in law school: the reading and comprehension of complex texts with accuracy and insight, the organization and management of information and the ability to draw reasonable inferences from it, the ability to think critically, and the analysis and evaluation of the reasoning and arguments of others.All of these changes are obviously a positive move, in that they are more likely to have greater precision for the perceived skill sets that the test intends to evaluate. The assumption that the exam achieves this goal has never been uniformly accepted, and continues to be reviewed by the profession and academia.
A new study on July 26, 2019 in the Journal of Empirical Legal Studies engaged in a longitudinal study of 1,400 students in 2 American law schools between 2005-2011. The authors found that LSAT and GPA are only weakly predictive of first-time bar passage rates, and that performance in the first year of law school had far greater predictive value. Upper year performance increased the predictive strength of the model. Despite the comments of some celebrities touting their intent to engage in at-home study of the law, this study suggests that legal education provides a significant impact on the understanding of the law by new lawyers, especially at a time when bar pass rates are declining in the U.S. This trend has also caused many law schools to reconsider their admissions criteria, including the LSAT.
A 2016 study by the same authors looked at included quality of major, undergraduate GPA trajectory, employment duration and type, leadership skills in college, graduate degrees, and prior criminal or disciplinary records, applying a multivariate regression analysis to predict law school grades. Some of the findings included that undergraduate GPA was a more powerful predictor than LSAT score, with a high LSAT and low GPA being a worse predictor than the opposite. Older studies also suggested that the LSAT was only weakly predictive of success in law school.
A rising GPA is only beneficial if the student is right out of school. A background in science, technology, engineering, math (STEM) or economics, accounting, finance (EAF) was a significant plus, similar to several additional LSAT points. Work experience was useful, but more useful if it was teaching and less useful if it was in the military, and 4-9 years of work is most ideal. Perhaps not surprisingly, a criminal or disciplinary record was a significant negative, equivalent to a larger equivalent on LSAT points. This study also found that several factors that have historically been considered by admissions as an advantage were not borne out by the data.
Common pre-law majors such as political science and history had no predictive value. Work experience in the legal or public sector was also not beneficial. Leadership skills in college was similarly non-notable for predicting law school grades. The authors state that leadership roles are often evaluated in a binary manner, and may not properly reflect the emotional intelligence (EI) that it may be intended to measure.
A surprising finding was that while gender discrepancies seemed to be abated, significant racial disparities exist. The authors conducted several computations on this specific component of the data, and concluded,
...it seems likely that the racial disparity reflects something not merely about the students, but about legal education itself—which may be unsurprising, given the substantial literature on how people of color, and those with less privileged socioeconomic backgrounds, can find law school alienating or a challenging adjustment, to the detriment of their performance. A full survey of the literature on alienation, stereotype threat, and other similar phenomena is beyond the scope of this article—but such phenomena are well-documented and long-known. Lani Guinier noted two decades ago, from survey and academic performance data, that women, then a minority of law students, found law school a source of “alienat[ion]” and “distress”—and performed worse in law school despite credentials on par with those of men:Of course this data only reflects the experience of two law schools, both situated in America, and may be differentiated in numerous ways from Canadian legal education. The authors also provide key caveats to their data, including that law school grades are themselves poor predictions of performance of lawyers, including their contributions to society, career fulfillment, and even job prospects in the long-term. A 2009 study of students who failed the American bar exam found that despite facing enormous challenges early on in their careers, these lawyers eventually spring back and even potentially out-perform the average college graduate in earnings.
[W]e find strong academic differences between graduating men and women. Despite identical entry-level credentials, this performance differential [is] ... maintained over ... [all] three years. By the end of their first year ..., men are three times more likely than women to be in the top 10% of their law school class.If anything, it is surprising that we found only racial disparities, not the gender disparities Guinier documented. Our findings thus evidence progress in eliminating law school gender disparities, but not racial disparities—warranting further support for struggling or alienated students...
Talent assessment is always more of an art than a science, but the hope of the use of empiricism of big data in law school admissions is that they will displace the often used anecdotes and preferences frequently used that are not substantiated by objective analysis of the information.
In the Journal of Law and Education last year, Aaron Taylor reviewed data from 16,000 law student respondents to Law School Survey of Student Engagement (LSSSE) to demonstrate how the notion of merit, as increasingly defined by Law School Admission Test (LSAT) scores, ensure that scholarships actually result in being directed to those with the most privileged backgrounds, with a notable racial element to them. He states that significant disparities exist in LSAT scores along ethnic, racial, and socioeconomic lines,
Inequality is an unpopular notion,' but it is tolerated as an unavoidable and largely unassailable reflection of varying talents and efforts.' This belief is the crux of American achievement ideology. Everyone gets what she deserves. The "haves" are lauded for their talents and "vision;" the "have-nots" are viewed in less flattering lights.' Moreover, these value-laden perceptions of success and failure are often internalized."o To use a common baseball analogy, individuals born on third base think they hit a homerun when they attain success, while those not even allowed an at-bat fault themselves for not rounding the bases. The dogma of achievement ideology crowds out discussions of structural factors that forestall equal opportunity." The outcome is an inequitable meritocracy that is nonetheless presumed to be a just means of selecting society's winners and losers. ... The LSAT-driven nature of scholarship awarding ensures that tuition discounts will flow most lucratively to students from privileged backgrounds. Disadvantaged students are then required to pay higher proportions of the tuition rate in order to account for the discounted rates of their privileged peers. While this method of cost-shifting is pervasive throughout legal education, the wealthiest institutions tend to rely less on tuition revenue in funding scholarships. For instance, Ivy League law schools award based predominantly on financial need. But because disadvantaged students are less likely to attend the wealthiest schools, they are disproportionately attending the most egregious adherents to the "Reverse Robin Hood" strategy. These trends, in turn, contribute to higher student loan debt among people least likely to afford to repay that debt.In other words, the effect of LSAT-based scholarships is that there is an economic subsidization of legal education by those who can least afford it. This disparity across different enumerated grounds have been the basis for human rights complaints in Canada, albeit remarkably unsuccessfully.
Challenges to the use of the LSAT, which is obviously heavily language based, have even been attempted against law schools on the basis of country of origin. In White v. University of Ottawa, Adjudicator Joachim stated,
 Professor Emard-Chabot testified that he reviewed the applicant’s entire application file, which included the written part of the LSAT In light of her legal education and experience, he thought she was an interesting candidate who would add to the diversity of the law school; however, her LSAT score, which was in the lowest one percentile, was troubling. He testified that the ability to read a large volume of material in English and write in English are important indicators of ability to complete the program. While the LSAT is not an exact predictor of success in law school, it is a factor that is taken into account. The extremely low LSAT score, the applicant’s grade of 64 in the only university level course she had taken, and the answer she provided in the written part of the LSAT test indicated to the Assistant Dean that the applicant would likely not be successful in law school. Accordingly, he made the determination, in conjunction with the admissions officer, to reject her application.[emphasis added]What is possibly more challenging is assessing the extent of accommodations based on disability. One of the accommodations that has the most significant impact with disabilities is flexible or extended time. However, given that the very nature of the LSAT is performance under time, LSAC does not readily provide this additional time without a clear basis for doing so. Last year, a California court held LSAC in contempt for their procedures for handling disability accommodations, specifically for additional time. The number of people seeking accommodations for disability have tripled since 2014, when LSAC entered into a consent decree that included cessation of flagging accommodated exam results.
Ruth Kolker argues in the Seton Hall Law Review that speeded exams should not be used for admissions regardless unless the time limits can be validated, given their impact on people with disabilities. She challenges the assumptions that faster performance is necessarily better, especially since approximately a fifth of LSAT writers do not complete the exam at all, and points to the experience of Pearson's Stanford 10 exam.
The removal of time limits better evaluated the skills and learning of students, and that the removal of these time limits did not result in unduly lengthy times for completion. She states,
The emphasis on rank-ordered scoring through a speeded testing instrument helps testing entities provide rankings at the top of the scale among applicants who are not meaningfully distinguishable on the basis of their test scores alone, at the expense of many applicants who do not even finish the exam. ...the fastest test taker may not necessarily be the best lawyer. In order to proceed quickly through a test such as the LSAT, a test taker needs to be willing to mark the right answer as soon as he or she spots it, proceeding through the A to E options. If A seems correct, the test taker needs to mark "A" and not even read the other options, but a good lawyer would rarely act so hastily. A good lawyer would want to read each option and double-check her answer in order not to make an error on behalf of a client. A good lawyer reads dense material word by word, looking for internal contradictions and inconsistencies. A good lawyer would wonder if more than one answer might actually be correct, rather than circling the first seemingly correct answer that is found. The "rush to complete" kind of speed embedded in all sections of the LSAT may actually be antithetical to the practice of law. But the LSAT, of course, only purports to predict performance in the traditional version of the first year of law school-a set of speeded exams, not the actual practice of law. The use of speeded exams in law school is not grounded in any formal or well defined theory. Rather, it is driven by a variety of nontheoretical factors, including the need to generate a grading curve, the desire to limit the volume of words that must be read, to enforce a consistent time limit, and the simple adherence to tradition.Kolker explores some of the rationales that are used to justify additional time for people with disabilities, but also notes the challenges in determining with precision the amount of extended time required to provide an equivalent score to non-accommodated test takers. Instead, she proposes a universal design solution that allows all test takers to take an exam under non-speeded conditions, which is grounded in best practices of disability studies in providing valid testing instrumentation for all groups, rather than different testing requirements for different groups. Universal Design for Learning (UDL) would have the additional benefit of assisting individuals with non-diagnosed disabilities, as well as applicants who have historically performed more poorly on the LSAT for reasons such as socio-economic background or other factors.
The flexibility employed in UDL assessments are more adaptive to students' needs around stress and anxiety. Instead of being geared towards the average student using a single method, UDL seeks greater clarity about what is actually being assessed. The most "speeded" section of the LSAT is the "analytical reasoning" section, also known as "logic games," which does not necessarily test any particular skill. Instead, it tests the ability of the test taker to implement certain shortcuts, which would have been learned prior to the exam.
Kolker notes that these shortcuts are not likely to be used in law school or as a practicing lawyer, meaning that the LSAT is really a test of whether they took the time to learn these shortcuts. The availability of expensive preparation courses is not necessarily available to all law school applicants in the same way, thereby excluding them from competitive consideration, especially if the pre-law period is characterized by the need to work and collect savings, instead of dedicated time to learning the shortcuts. LSAT scores therefore in large measure reflect and amplify socio-economic circumstances far more than they do raw skill or talent. LSAC's policies have also been litigated in Canada over the years, applying additional pressure for reform.
A flexible approach, similar to what is now currently being implemented by LSAC, appears to be successful in handling these cases appropriately. The backdrop to these changes have been contentions ones.
In Arenson v. Law School Admission Council, the Tribunal rejected the jurisdiction argument by LSAC in 2010, who claimed that they provide their services out of America and should not be subject to the human rights regime in Ontario. The parties subsequently confirmed a Minutes of Settlement, demonstrating considerable latitude by LSAC to accommodate the applicant's learning disability, and foreshadowing some of the accommodations that LSAC would later implement. This applicant subsequently was successful in gaining admission to a Canadian law school, and was subsequently a judicial law clerk. The outcome of these accommodations for disability may not necessarily always match the desired extent of flexibility by an applicant.
A greater challenge is where an applicant alleges discrimination directly against the educational institution. In Ruffolo v. York University, the applicant sought to have the LSAT score excluded entirely from his application, but was dismissed on a section 53(3) of the Human Rights Code motion stage as being plain and obvious as certain to fail.
More recently, the Saskatchewan Court of Appeal in Yashcheshen v University of Saskatchewan conducted a judicial review of a decision, which denied the law school applicant's request to have her LSAT score excluded from consideration. LSAC appears to have granted certain accommodations based on the applicant's Crohn’s disease, hypothyroidism and iritis, but apparently denied other forms, including the use of marijuana during testing and stopping the clock during breaks. The refusal to then exclude the LSAT from the law school application was the alleged basis of discrimination. This medical request of LSAC may not be as strange as it appears, as there are some scientific indications of promising therapeutic benefit of marijuana for these conditions.
In 2004, the B.C. Human Rights Tribunal in Carlisle v. Law School Admission Council dealt with a similar request to use marijuana for a physical disability, but the tribunal focused exclusively on the jurisdiction issue, coming to the same conclusion as noted above. The case appears to have settled privately thereafter.
In Yashcheshen, the Court of Appeal focused narrowly on whether the Charter applies to the law school's LSAT policy, largely because the applicant had initiated separate proceedings in the Human Rights Commission which were never continued, and was not seeking judicial review of those proceedings (despite including their content before the court).
The court engaged in a rather traditional analysis of the applicability of the Charter to universities, concluding that s. 15 did not apply to the LSAT dimension of the College’s admissions policy. The court did state,
 In saying all of this, I appreciate that there is a line of cases where the Charter has been applied to actions of universities when those actions have involved the exercise of what might be called statutorily-based powers of compulsion. See, for example: Pridgen v University of Calgary, 2012 ABCA 139 (CanLII), 350 DLR (4th) 1 (student discipline); R v Whatcott, 2014 SKPC 215 (CanLII), 464 Sask R 105 (prohibition of a demonstration); and R v Whatcott, 2002 SKQB 399 (CanLII),  4 WWR 149(prohibition of pamphleting). These authorities do not deal with situations parallel to the one at hand and it is therefore not necessary to consider them in order to resolve this appeal.The 2017 Ryerson law school application indicates that in order to encourage the broadest number of candidates from many diverse backgrounds in applying, the school does not intend to advertise suggested LSAT or GPA ranges. This provides the admissions committee the greatest flexibility in evaluating candidates' potential. However, some of the human rights cases in Canada suggest that this data may in fact be compelled, if there is an application on the basis of human rights, so this may become part of the public domain.
The new format of the LSAT should assist in applying it in a more flexible manner, especially when dealing with accommodation requests based on disabilities. Whether it should remain a primary and a significant driver for law school applications remains an ongoing question. Increasingly, the data is demonstrating that LSAT scores provide an incomplete picture of an applicant's potential, especially in the Canadian context where bar exam pass rates are not the primary metric used to evaluate pedagogical success within law school.
While the LSAT will invariably continue to be used in Canadian law schools in the near future, the weight provided to it should be carefully balanced with other factors, and with careful reflection of the characteristics we seek in the next generation of lawyers.