Measuring revealed student scheduling preferences using constrained discrete choice models

J. Bailey, M. West, and C. Zilles

in Proceedings of the 124th American Society for Engineering Education Annual Conference and Exposition (ASEE 2017), Paper ID #19940, 2017.

For constrained student resources with large student populations it is often necessary to implement some form of reservation or scheduling system. Examples of scheduled-access resources can include one-on-one tutoring, machine shops or labs, and computer-based testing facilities. For planning and resource scheduling purposes it is important to be able to forecast demand, and thus it is important to understand what drives student preferences for particular scheduling time slots. Measuring these preferences can be challenging, however, for at least the following three reasons. (1) Revealed preferences (what students actually choose) can differ significantly from stated preferences (what they say they will want at a future time), requiring the use of actual scheduling data to infer preferences or utilities. (2) The utility that students derive from particular choices is multifactorial, so that in a computer-based testing facility, for example, students may prefer to take their exam mid-afternoon, but they may also prefer to take it as close to the end of the exam period as possible, and it can be difficult to disentangle these factors. (3) Capacity constraints will frequently lead to many time slots being fully reserved, making it unclear which slots were actually preferred. This paper presents a general framework for measuring revealed student preferences from actual reservation or scheduling data. This framework is based on the theory of constrained discrete choice modeling, as used in economics for modeling consumer preferences. A multifactorial random utility model (RUM) is formulated for student scheduling preferences and the model is trained on scheduling data using maximum likelihood estimation (MLE) and cross-validated on multiple rounds of training/test data splits. Results are presented using scheduling data from a computer-based testing facility with approximately 50,000 student reservations over three semesters (Spring 2015 to Spring 2016, inclusive). We show that this measurement methodology can accurately capture student preferences in real-world scheduling data, and can successfully separate out time-in-week preferences from time-within-exam preferences. Errors are quantified using both log-likelihood with per-reservation data and root mean square error (RMSE) with data aggregated to the time slot level. We discuss both estimation and simulation algorithms for constrained discrete choice models and show how Monte Carlo simulation can be used to obtain uncertainty predictions for predicting expected usage.

External link: https://peer.asee.org/28656

Full text: BaWeZi2017.pdf