This case originated from a class action lawsuit filed by the Center for Independence of the Disabled, New York and other nonprofit organizations (collectively referred to as Plaintiffs) against the Metropolitan Transportation Authority (“MTA), New York City Transit Authority (“NYC Transit”), and the City of New York(“the City”) (collectively referred to as Defendants). The lawsuit alleged that hundreds of thousands of New Yorkers with mobility disabilities were persistently and discriminatorily excluded from accessing the New York City subway system due to the failure of the Defendants to adequately maintain the already limited number of elevators in the system.

The Plaintiffs contended that subway riders with mobility disabilities routinely faced abrupt and extended elevator outages lasting for months, without any prior notice or warning mechanisms like signage or announcements. Moreover, the Defendants allegedly failed to provide any alternative accommodations to facilitate the transportation of people requiring elevator access when outages occurred. The Plaintiffs asserted that the overall elevator accessibility in the New York City subway system was already poor compared to other major US cities, with only 112 (24%) of the 472 subway service line stations in New York City being wheelchair-accessible.

Public data from the MTA demonstrates that over 9,019 elevator outages occurred during the one-year period ending on June 30, 2015. Over 4,100 of those outages were unscheduled—a predictable result of Defendants’ failure to maintain and implement adequate preventative maintenance procedures

Plaintiffs alleged that Defendants’ failure to maintain the limited number of elevators they provide in the New York City subway system violated federal and local disability rights laws, including Title II of the Americans with Disabilities Act (“ADA”), Section 504 of the Rehabilitation Act of 1973, and the New York City Human Rights Law (“NYCHRL”). The Court was left to assess whether Defendants have established that they provide reasonable accommodations to class members when elevator outages prevent them from accessing the subway.

The Defendants retained Dr. Alan J. Salzberg, a statistician, to analyze the overall median availability of elevators in the NYC subway system using MTA data. The Plaintiffs did not contest his availability calculations but argued that the aggregated figures could obscure variability. Hence, the Plaintiffs presented two rebuttal experts – Andrew D. Schwarz and Sylvia Morse – to provide context to Dr. Salzberg’s opinions.

The Plaintiffs also filed a motion to exclude Dr. Salzberg’s testimony, arguing that he exceeded his expertise scope, manipulated data unreliably, and made flawed assumptions, rendering his testimony unhelpful and misleading.

Statistics Expert Witness

Alan J. Salzberg Ph.D., is Senior Statistician and Principal of Salt Hill. His focus is statistical analysis, sampling, estimation, and modeling, especially using large or complex datasets. Many of Dr. Salzberg’s consulting projects and research papers have related to the detection and measurement of bias. He has testified as an expert witness in statistics in federal and state court. Prior to joining Salt Hill, Alan was CEO of Analysis & Inference. Salzberg holds a Ph.D. in Statistics from the Wharton School of the University of Pennsylvania, where he also received a Bachelor of Science in Economics.

Discussions by the Court

In response to Plaintiffs’ motion to exclude the testimony of Defendants’ expert, Alan Salzberg, the Court engaged in a detailed discussion rooted in the parameters set by Federal Rule of Evidence 702. The Court emphasized that, under Rule 702, the party seeking to introduce expert testimony bore the burden of demonstrating that the testimony met certain criteria, including being based on sufficient facts or data, the product of reliable principles and methods, and reliably applied to the facts of the case. Additionally, the expert’s testimony had to be relevant and assist the jury in understanding the case.

The Court considered the motion to exclude the testimony of Alan Salzberg, the Defendants’ expert, who had submitted reports critiquing two of the Plaintiffs’ experts, Andrew Schwarz, an economist, who developed a model to demonstrate the practical implications of elevator outages on passengers traveling on certain high-usage subway routes during rush hours, and Sylvia Morse, Policy Program Manager at Pratt Center for Community Development. The Court began by addressing Salzberg’s expert report titled “Subway Accessibility Analysis.” Salzberg, was tasked with reviewing public elevators in the New York City subway system to determine the elevator, station, and station stop availability for riders with mobility disabilities. Schwarz’s analysis was “intended to highlight the impact of the frequency of inaccessibility calculated by Salzberg on the commutes of targeted types of individuals.”

In response to Schwarz’s report, Salzberg submitted a reply report on May 2019, using the same Metropolitan Transportation Authority’s (“MTA”) Trip Planner tool that Schwarz relied on for his analysis. Salzberg performed his own analysis of the 200 trips considered by Schwarz and identified errors in Schwarz’s analysis, such as, Schwarz had not selected the 200 subway trips at random and excluded downtown Manhattan stations.

After the Second Circuit’s remand, Salzberg submitted an update to his report, dated September 12, 2022, concerning accessible elevator, station, and station stop availability, using new data, from 2019 through 2021. In response, Plaintiffs submitted an updated report from Schwarz, dated November 23, 2022. Schwarz, using Salzberg’s statistical calculations, again examined the likelihood that a commuter taking a trip along 200 commuting routes involving subway stations commonly used by passengers with mobility disabilities would experience a failed trip during a particular period of time. The Court noted that Plaintiffs did not challenge Salzberg’s availability calculations, focusing instead on his qualifications and the assumptions underlying his statistical calculations.

Plaintiffs argued that Salzberg was unqualified to render an expert opinion in the case due to his lack of expertise in transportation systems, commuting patterns in New York City, or accessibility barriers for people with mobility disabilities. The Court countered this by highlighting that Salzberg analyzed the same data examined by Schwarz, who himself was an economist without expertise in transportation systems. The Court emphasized that Salzberg’s qualifications were appropriate for critiquing Schwarz’s model.

Another argument raised by Plaintiffs was the unreliability of Salzberg’s methodologies, grounded in alleged flawed assumptions. Plaintiffs contended that these flaws rendered his opinion unreliable and unhelpful to the trier of fact. The Court, however, clarified that criticisms of methodology went to the weight of the expert’s opinion, not its admissibility. The Court stated that faults in methodology were a matter for cross-examination.

Plaintiffs specifically challenged Salzberg’s assumption that the MTA’s Trip Planner accurately reflected availability and delays associated with having to reroute or that passengers received timely notification of elevator outages. The Court deemed it reasonable for Salzberg to have relied on data pertaining to route availability and delays, which were maintained by the MTA in the regular course of business and were accessible to the public through the MTA’s Trip Planner online tool. The Court also noted Schwarz’s reliance on the same Trip Planner data in his analysis, highlighting that any inaccuracies would affect both analyses. Plaintiffs’ claims about data inaccuracy were considered arguments concerning the weight of the evidence and not to its admissibility.

Another point of contention was Salzberg’s definition of a “successful” trip, where Plaintiffs argued that he deemed a trip successful regardless of its duration or onerousness. Salzberg’s use of the Trip Planner tool to determine alternative itineraries was defended by the Court, emphasizing that Plaintiffs could challenge the feasibility of these alternatives on cross-examination.

The Court also addressed Salzberg’s consideration of nearby stations when an elevator outage affected the original station, adding travel time to the itinerary whenever necessary. Plaintiffs contended that this introduced selection bias. The Court rejected this argument, stating that Salzberg reliance on the MTA’s Trip Planner to determine alternative itineraries demonstrated no selection bias.

Moving to the critique of Plaintiffs’ expert Sylvia Morse, the Court outlined the situation. Morse had submitted a rebuttal report focusing on the real-world performance of the bus system and
factors affecting rider experience, challenging two assumptions made by Salzberg. The first assumption was that transit users with mobility disabilities could always access buses, and the second was that the MTA’s Trip Planner accurately reflected a transit user’s travel time. Ultimately, Morse concluded that accessibility barriers and the unreliability of buses impeded or delayed the completion of alternative itineraries that relied on use of the bus system for transit users with mobility disabilities.

In response, Salzberg prepared a rebuttal report criticizing Morse’s reliance on anecdotal customer complaints and arguing that such complaints represented a tiny fraction of all accessible bus trips and was not a statistically valid basis to support a conclusion that the bus system suffered from widespread barriers to accessibility. Plaintiffs challenged Salzberg’s analysis, raising arguments that the Court deemed relevant to the weight of the evidence, not its admissibility.

Plaintiffs faulted Salzberg for assuming that each complaint concerned only a single ride or bus operator. The Court noted that Plaintiffs had no evidence to support their assumption that a complaint could pertain to multiple trips, highlighting the lack of basis for this criticism. Plaintiffs’ challenge to Salzberg’s conclusion regarding Trip Planner’s notification of elevator outages in real time was also addressed. The Court acknowledged Plaintiffs’ freedom to challenge the persuasiveness of this opinion on cross-examination but found the assumption not unfounded enough to render the opinion inadmissible.

Held

The Court denied the Plaintiffs’ Daubert motion to exclude Dr. Salzberg’s testimony, finding that their criticisms went to the weight rather than the admissibility of his opinions. The Court determined there was no basis to preclude Dr. Salzberg’s expert testimony under the applicable rules of evidence. The Court has not arrived on an outcome for this case since the remaining issues involved in this case still await resolution.

Key Takeaways

This case highlights several important considerations regarding the admissibility of expert witness testimony. A key takeaway is that critiques of an expert’s methodology ,sometimes, generally relate to the weight of the testimony rather than its admissibility. The Court emphasized that alleged flaws in an expert’s assumptions or analysis are fodder for cross-examination but do not necessitate exclusion. Furthermore, the ruling demonstrates that rebuttal experts need not have specialized expertise in the substantive field at issue if they are commenting on another expert’s methodology. Finally, the Court highlighted that rebuttal experts should bring specialized knowledge to bear on an issue rather than just speculating or relying solely on anecdotal evidence. These insights shed light on key considerations for expert witness testimony.

Tag: Statistical analysis