Back To Schedule
Friday, November 15 • 4:20pm - 4:50pm
Build Your Own ML Data Feedback Loop

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Machine learning models should learn from their history. Data collection and labeling is often the rate-limiting step of AI research. At Curai, our AI tools are deployed in a real-world healthcare setting, giving us the opportunity to learn from their usage. This talk will focus on how to build a semi-automated data feedback loop for ML model retraining, highlighting the specific use case at Curai. A data feedback loop consists of several key components. First, model output is presented to the user (in our case, a doctor or health professional), who can choose to accept or reject a medical suggestion. This usage data is then sent to data sinks and forwarded to a data store, where post-processing and additional calculations can happen (for example, calculating the edit distance between two strings). Processed data can then be sent down (most simply, through a CSV) to a model for retraining or fine-tuning, and the resulting v2 model can then be tested for accuracy and re-deployed into the product. In short, the semi-automated data feedback loop allows for rapid iteration and continuous learning for AI/ML models. This talk will focus on specific technologies I and my teammates have used, including, but not limited to, integration with StackDriver, BigQuery, and LaunchDarkly. Attendees will learn how to build a semi-automated data feedback loop, practical code examples and anecdotes of my own failures and successes in this domain, and ethical implications of using user-generated data for model retraining. There is tremendous potential for AI in healthcare, and closing the data loop for model retraining can help solve one of the key challenges in this domain and continuously improve machine learning models.

avatar for Sophia Sanchez

Sophia Sanchez

Machine Learning Engineer, Curai

Friday November 15, 2019 4:20pm - 4:50pm PST