Jacques is a data scientist in London and founder of Stakion, a ML monitoring startup. Prior to starting Stakion, he was a data scientist for 4 years with a focus on getting models in production for both startups (Triptease, Brightmile) and large organizations (Worldpay). During that time, he built and deployed for payment optimization, credit card fraud detection, transport mode detection, etc.
Before transitioning to Data Science, Jacques studied Computational Fluid Dynamics (Imperial College London, Msc - UK) and Engineering (Ecole Centrale Paris, Msc - France).
AI has been migrating over the last couple of years from experimentation and proof of concepts to revenue-generating applications. As a result, organization is increasingly exposed to the risks associated with running AI systems in the real-world.
Nearly all AI systems share two fundamental characteristics: they rely on complex data pipelines and assume that the world is static. The assumption that the world is static leads to long term performance degradation as the world changes while complex data pipelines introduce the risk of breaking changes that will lead to abrupt drops in performance. This talk will start by describing a high-level Model Risk Management Framework before detailing some of the key features a Machine Learning monitoring solution should have.