Javascript must be enabled to continue!
Evaluating Human-AI Interfaces in Logistics: Impact on Cognitive Workload and Usability

Integrating Artificial Intelligence (AI) algorithms into logistics decision-support systems relieves operators of complex manual calculations (e.g., bin-packing, dynamic rerouting). However, opaque 'black-box' automation often leads to trust calibration failures, inducing either overtrust (automation bias) or undertrust (system disuse). This study investigates how an Explainable AI interface designed according to Microsoft Human-AI Experience (HAX) guidelines and participatory design principles impacts operator cognitive workload (NASA-TLX), perceived usability (SUS), time on task, operational errors, and calibrated trust compared to an unassisted baseline interface.

Implemented within a single-page application framework, the dispatcher workflow is structured into three sequential sub-tasks: (1) Order Intake, (2) Capacity Allocation, and (3) Crisis Recovery. We evaluated two interface paradigms: an unassisted Baseline UI and an AI-Enhanced UI.  Following participatory design principles, the baseline interface was co-designed with an active logistics professional with over 7 years of domain experience, alongside iterative feedback from 5 additional industry practitioners. The AI-Enhanced UI implements Microsoft HAX guidelines across interaction stages: clarifying capability boundaries (G1), communicating uncertainty via approximate ETA ranges (G2), providing proactive context-aware filtering (G3, G4), visualizing local explanations detailing vehicle suitability based on ETA and remaining cubic volume (G11-A), exposing override controls for human-in-the-loop governance (G1, G7), and employing session caching to avoid repeating rejected suggestions (G12).

A counterbalanced within-subjects user study was conducted with 22 professional logistics operators (). Empirical results revealed that the AI-enhanced UI significantly reduced perceived cognitive workload by  ( vs. , , ) and significantly lowered operational errors by  ( =   vs.  = , , r = 0.449). Perceived usability improved significantly from  to  on the SUS scale (,), elevating the interface to the 'Best Imaginable' category. Task completion time showed no significant difference ( vs. , ), reflecting a speed-accuracy tradeoff where operators actively reviewed local explanations rather than relying blindly on automated outputs. Subjective ratings confirmed high explainability (/ ) and balanced reliance (/ proving that HAX-grounded design fosters calibrated trust and preserves operator agency in critical logistics operations.

 

Attachment language: English File type: PDF document Presentation Poster
Updated: 02-11-2026 23:40 - Size: 4.31 MB
Attachment language: English File type: PowerPoint presentation Presentation File
Updated: 02-11-2026 23:44 - Size: 4.57 MB
Speakers
Evangelia FotidouLampros GiotisChristos Katsanos

Back

ACM GreekCHI-CYCHI Symposium 2026


We are excited to invite you to the ACM GreekCHI-CYCHI Symposium 2026, held on November 6-7, 2026 jointly in Athens, Thessaloniki, Nicosia and Online.