Web-based experimental economics software: How do they compare to desirable features?

Abstract

Web-based experiments that cut across the lab vs. field distinction are increasingly popular with economists. However, non-standardized software features and services hinder comparability and replication. This study reviews a wide selection of experimental economics software packages and evaluates them against criteria based on the logistics and operational requirements of economic experiments. We find that oTree and SoPHIE rank highest across criteria, but Veconlab and classEx might be suitable for those with a dominant need for a large library of ready-made experiments. We find a portability gap: no presently available software allows portability of experiments across platforms because of technical complexity and the challenging coordination needs of experimental economists. As a result, experiments may be replicated only on the same platform or with the same software, but general replicability is slow and costly. This constrains the development of experimental economics as a replicable science.