Bridging Big Data and Data Science Using Scalable Workflows - Ilkay Altintas
Scientific workflows are used by many scientific communities to capture, automate and standardize computational and data practices in science. Workflow-based automation is often achieved through a craft that combines people, process, computational and Big Data platforms, application-specific purpose and programmability, leading to provenance-aware archival and publications of the results. This talk will summarize varying and changing requirements for scalability in distributed workflows influenced by Big Data and heterogeneous computing architectures. It will also present our ongoing research efforts on end-to-end performance prediction for workflow-driven Big Data applications based on these maturing requirements.
By: Ilkay Altintas (SDSC)
If we knew what it was we were doing, it would not be called research, would it? - Albert Einstein