The work of developing cyberinfrastructure middleware projects

Middleware software, which provides an abstraction layer between low-level computational services and domain-specific applications, is a key component of cyberinfrastructure. This paper presents a qualitative study of how cyberinfrastructure middleware development is accomplished in two supercomputing centers. Our investigation highlights key development phases in the lives of middleware projects. Middleware development is typically undertaken as part of collaborations between technologists and domain scientists, and middleware developers must balance the pressure to meet specific scientific needs and the desire to explore their own R&D agendas. We explore how developers work to sustain an ongoing development trajectory by aligning their own work with particular domain science projects and funding streams. However, we find that the key transition from being a component in a domain-specific project to a stand-alone system that is useful across domains is particularly challenging for middleware development. We provide organizational and national policy implications for how to better support this transition.