Michal Valko
- Institut National de Recherche en Informatique et en Automatique (Inria)
- Ecole Normale Supérieure Paris-Saclay (ENS Paris Saclay)
- DeepMind [Paris]
- Meta AI Research [Paris]
Présentation
Michal is the Founding Researcher at a stealth startup, tenured researcher at Inria, and a lecturer at MVA at ENS Paris-Saclay. Michal is primarily interested in designing algorithms that would require as little human supervision as possible. He works on methods and settings that are able to deal with minimal feedback, such as deep reinforcement learning, bandit algorithms, self-supervised learning, or self play. Michal has recently worked on representation learning, world models and deep (reinforcement) learning algorithms that have some theoretical underpinning. In the past he has also worked on sequential algorithms with structured decisions where exploiting the structure leads to provably faster learning. Michal is now working on a new generation of large language models (LLMs), in addition to providing algorithmic solutions for their scalable test-time inference, fine-tuning and alignment. He received his PhD in 2011 from the University of Pittsburgh, before getting a tenure at Inria in 2012 and co-creating Google DeepMind Paris with R. Munos. In 2024, he became a Principal Llama Scientist at Meta, building online reinforcement learning stack and research for Llama 3.
Domaines de recherche
Publications
Publications
|
|
The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit FeedbackICML 2025 - 42nd International Conference on Machine Learning, Jul 2025, Vancouver, Canada |
|
|
A general theoretical paradigm to understand learning from human preferencesAISTATS 2024 - 27th International Conference on Artificial Intelligence and Statistics, May 2024, Valencia, Spain |
|
|
Decoding-time realignment of language modelsICML 2024 - 41st International Conference on Machine Learning, Jul 2024, Vienna, Austria |
|
|
Human alignment of large language models through online preference optimisationICML 2024 - 41st International Conference on Machine Learning, Jul 2024, Vienna, Austria |
|
|
Unlocking the power of representations in long-term novelty-based explorationICLR 2024 - 12th International Conference on Learning Representations, May 2024, Vienna, Austria |
rlberry - A Reinforcement Learning Library for Research and EducationMachine Learning Conference, Jan 2024, Online, United States |
|
|
|
Generalized preference optimization: A unified approach to offline alignmentICML 2024 - 41st International Conference on Machine Learning, Jul 2024, Vienna, Austria |
|
|
Demonstration-Regularized RLThe Twelfth International Conference on Learning Representations, May 2024, Vienne, Austria. ⟨10.48550/arXiv.2310.17303⟩ |
|
|
Metacognitive capabilities of LLMs: An exploration in mathematical problem solvingNeurIPS 2024 - Annual Conference on Neural Information Processing Systems, Dec 2024, Vancouver, Canada |
|
|
Nash learning from human feedbackICML 2024 - 41st International Conference on Machine Learning, Jul 2024, Vienna, Austria |
|
|
Quantile credit assignmentInternational Conference on Machine Learning, Jul 2023, Honolulu, United States |
|
|
Model-free Posterior Sampling via Learning Rate RandomizationAdvances in Neural Information Processing Systems 36 (NeurIPS 2023), Dec 2023, New Orleans, United States. ⟨10.48550/arXiv.2310.18186⟩ |
|
|
Understanding self-predictive learning for reinforcement learningICML 2023 - 40th International Conference on Machine Learning, Jul 2023, Honolulu, United States |
|
|
DoMo-AC: Doubly multi-step off-policy actor-critic algorithmICML 2023 - 40th International Conference on Machine Learning, Jul 2023, Honolulu, United States |
|
|
Regularization and variance-weighted regression achieves minimax optimality in linear MDPs: Theory and practiceICML 2023 - International Conference on Machine Learning, Jul 2023, Honolulu, United States |
|
|
Curiosity in hindsight: Intrinsic exploration in stochastic environmentsICML 2023 - 40th International Conference on Machine Learning, Jul 2023, Honolulu, United States |
|
|
VA-learning as a more efficient alternative to Q-learningInternational Conference on Machine Learning, Jul 2023, Honolulu, United States |
|
|
Half-Hop: A graph upsampling approach for slowing down message passingICML 2023 - 40 th International Conference on Machine Learning, Jul 2023, Honolulu, United States |
|
|
BYOL-Explore: Exploration by bootstrapped predictionNeurIPS 2022 - 36th International Conference on Neural Information Processing Systems, Nov 2022, New Orleans, United States |
|
|
Retrieval-augmented reinforcement learningInternational Conference on Machine Learning, Jul 2022, Baltimore, United States |
|
|
From Dirichlet to Rubin: Optimistic exploration in RL without bonusesICML 2022 - 39th International Conference on Machine Learning, Jul 2022, Baltimore, United States |
|
|
Large-scale representation learning on graphs via bootstrappingInternational Conference on Learning Representations, Apr 2022, Virtual, United States |
|
|
Scaling Gaussian process optimization by evaluating a few unique candidates multiple timesInternational Conference on Machine Learning, Jul 2022, Baltimore, United States |
|
|
Adaptive multi-goal explorationAISTATS 2022 - 25th International Conference on Artifi- cial Intelligence and Statistic, Mar 2022, Virtual, Spain |
|
|
Marginalized operators for off-policy reinforcement learningAISTATS 2022 - 25th International Conference on Artifi- cial Intelligence and Statistics, Mar 2022, Virtual, Spain |
|
|
Mine Your Own vieW: Self-supervised learning through across-sample predictionICML 2021 - The 38th International Conference on Machine Learning, Jan 2021, Online, United States |
|
|
Revisiting Peng's Q(λ) for for modern reinforcement learningInternational Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria |
|
|
Online A-optimal design and active linear regressionInternational Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria. ⟨10.48550/arXiv.1906.08509⟩ |
|
|
Episodic reinforcement learning in finite MDPs: Minimax lower bounds revisitedAlgorithmic Learning Theory, Mar 2021, Paris / Virtual, France |
|
|
Unifying gradient estimators for meta-reinforcement learning via off-policy evaluationNeural Information Processing Systems, Dec 2021, Virtual, United States |
|
|
Fast active learning for pure exploration in reinforcement learningInternational Conference on Machine Learning, Jul 2021, Vienna, Austria |
|
|
Drop, Swap, and Generate: A self-supervised approach for generating neural activityNeural Information Processing Systems, Dec 2021, Virtual, United States |
|
|
Kernel-based reinforcement Learning: A finite-time analysisInternational Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria |
|
|
Model-free learning for two-player zero-sum partially observable Markov games with perfect recallNeural Information Processing Systems, Dec 2021, Virtual, United States |
|
|
Broaden your views for self-supervised video learningIEEE/CVF International Conference on Computer Vision, Oct 2021, Virtual, Canada |
|
|
A kernel-based approach to non-stationary reinforcement learning in metric spacesInternational Conference on Artificial Intelligence and Statistics, Apr 2021, San Diego / Virtual, United States |
|
|
Taylor expansion of discount factorsInternational Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria |
|
|
Sample complexity bounds for stochastic shortest path with a generative modelAlgorithmic Learning Theory, 2021, Paris, France |
|
|
UCB Momentum Q-learning: Correcting the bias without forgettingInternational Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria |
|
|
Density-based bonuses on learned representations for reward-free exploration in deep reinforcement learningICML 2021 - The 38th International Conference on Machine Learning, Jan 2021, Online, United States |
|
|
Adaptive reward-free explorationAlgorithmic Learning Theory, 2021, Paris, France |
|
|
Covariance-adapting algorithm for semi-bandits with application to sparse outcomesConference on Learning Theory, 2020, Graz, Austria |
|
|
Taylor expansion policy optimizationInternational Conference on Machine Learning, 2020, Vienna, Austria |
|
|
Adaptive multi-fidelity optimization with fast learning ratesInternational Conference on Artificial Intelligence and Statistics, 2020, Palermo, Italy |
|
|
Statistical efficiency of Thompson sampling for combinatorial semi-banditsNeural Information Processing Systems Conference, Dec 2020, Virtual, France |
|
|
Near-linear time Gaussian process optimization with adaptive batching and resparsificationInternational Conference on Machine Learning, 2020, Vienna, Austria |
|
|
Monte-Carlo tree search as regularized policy optimizationInternational Conference on Machine Learning, 2020, Vienna, Austria |
|
|
Bootstrap Your Own Latent: A new approach to self-supervised learningNeural Information Processing Systems, 2020, Montréal, Canada |
|
|
Fixed-confidence guarantees for Bayesian best-arm identificationInternational Conference on Artificial Intelligence and Statistics, 2020, Palermo, Italy |
|
|
Gamification of pure exploration for linear banditsICML 2020 - International Conference on Machine Learning, Aug 2020, Vienna / Virtual, Austria |
|
|
Derivative-free & order-robust optimisationInternational Conference on Artificial Intelligence and Statistics, Aug 2020, Palermo / Virtual, Italy |
|
|
No-regret exploration in goal-oriented reinforcement learningInternational Conference on Machine Learning, 2020, Vienna / Virtual, Austria |
|
|
Sampling from a k-DPP without looking at all itemsNeural Information Processing Systems, 2020, Montréal, Canada |
|
|
Planning in Markov Decision Processes with Gap-Dependent Sample ComplexityNeural Information Processing Systems, 2020, Vancouver, France |
|
|
A single algorithm for both restless and rested rotting banditsInternational Conference on Artificial Intelligence and Statistics, Aug 2020, Palermo / Virtual, Italy |
|
|
Budgeted online influence maximizationInternational Conference on Machine Learning, 2020, Vienna, Austria |
|
|
Stochastic bandits with arm-dependent delaysInternational Conference on Machine Learning, 2020, Vienna, Austria |
|
|
Reward-free exploration beyond finite-horizonICML 2020 Workshop on Theoretical Foundations of Reinforcement Learning, 2020, Vienna, France |
|
|
Improved sample complexity for incremental autonomous exploration in MDPsNeural Information Processing Systems, 2020, Montréal, Canada |
|
|
BYOL works even without batch statisticsNeurIPS 2020 Workshop: Self-Supervised Learning - Theory and Practice, Jan 2020, Online, United States |
|
|
Improved sleeping bandits with stochastic action sets and adversarial rewardsICML 2020 - International Conference on Machine Learning, Jul 2020, Vienna, Austria |
|
|
On two ways to use determinantal point processes for Monte Carlo integration -- Long versionNeurIPS 2019 - Thirty-third Conference on Neural Information Processing Systems, Jun 2019, Vancouver, Canada |
|
|
A simple dynamic bandit algorithm for hyper-parameter tuningWorkshop on Automated Machine Learning at International Conference on Machine Learning, AutoML@ICML 2019 - 6th ICML Workshop on Automated Machine Learning, Jun 2019, Long Beach, United States |
|
|
A simple parameter-free and adaptive approach to optimization under a minimal local smoothness assumptionAlgorithmic Learning Theory, 2019, Chicago, United States |
|
|
Gaussian process optimization with adaptive sketching: Scalable and no regretConference on Learning Theory, 2019, Phoenix, United States |
|
|
Exact sampling of determinantal point processes with sublinear time preprocessingNeural Information Processing Systems, 2019, Vancouver, Canada |
|
|
Exploiting structure of uncertainty for efficient matroid semi-banditsInternational Conference on Machine Learning, 2019, Long Beach, United States |
|
|
Finding the bandit in a graph: Sequential search-and-stop22nd International Conference on Artificial Intelligence and Statistics (AISTATS 2019), Apr 2019, Okinawa, Japan |
|
|
Active multiple matrix completion with adaptive confidence setsInternational Conference on Artificial Intelligence and Statistics, 2019, Okinawa, Japan |
|
|
Rotting bandits are not harder than stochastic onesInternational Conference on Artificial Intelligence and Statistics, 2019, Naha, Japan |
|
|
Les processus ponctuels déterminantaux en apprentissage automatiqueGRETSI 2019 - XXVIIe Colloque sur le Traitement du Signal et des Images, Jan 2019, Online, France |
|
|
General parallel optimization without a metricAlgorithmic Learning Theory, 2019, Chicago, United States |
Optimizing human learning workshop eliciting adaptive sequences for learning (WeASeL)Machine Learning Conference, Jan 2019, Online, United States |
|
|
|
Multiagent evaluation under incomplete informationNeurIPS 2020 - 33rd Conference on Neural Information Processing Systems, Dec 2019, Vancouver, Canada |
|
|
Planning in entropy-regularized Markov decision processes and gamesNeural Information Processing Systems, 2019, Vancouver, Canada |
|
|
Scale-free adaptive planning for deterministic dynamics & discounted rewardsInternational Conference on Machine Learning, 2019, Long Beach, United States |
|
|
Improved large-scale graph learning through ridge spectral sparsificationInternational Conference on Machine Learning, Jul 2018, Stockholm, Sweden |
|
|
Optimistic optimization of a BrownianNeurIPS 2018 - Thirty-second Conference on Neural Information Processing Systems, Dec 2018, Montréal, Canada |
|
|
Adaptive black-box optimization got easier: HCT only needs local smoothnessEuropean Workshop on Reinforcement Learning, Oct 2018, Lille, France |
|
|
Best of both worlds: Stochastic & adversarial best-arm identificationConference on Learning Theory, 2018, Stockholm, Sweden |
|
|
Compressing the Input for CNNs with the First-Order Scattering TransformECCV 2018 - European Conference on Computer Vision, Sep 2018, Munich, Germany |
|
|
Efficient second-order online kernel learning with adaptive embeddingNeural Information Processing Systems, 2017, Long Beach, United States |
|
|
Online influence maximization under independent cascade model with semi-bandit feedbackNeural Information Processing Systems, Dec 2017, Long Beach, United States. pp.1-24 |
|
|
Second-Order Kernel Online Convex Optimization with Adaptive SketchingInternational Conference on Machine Learning, 2017, Sydney, Australia |
|
|
Zonotope hit-and-run for efficient sampling from projection DPPsInternational Conference on Machine Learning, 2017, Sydney, Australia |
|
|
Distributed adaptive sampling for kernel matrix approximationInternational Conference on Artificial Intelligence and Statistics, 2017, Fort Lauderdale, United States |
|
|
Trading off rewards and errors in multi-armed banditsInternational Conference on Artificial Intelligence and Statistics, 2017, Fort Lauderdale, United States |
|
|
Blazing the trails before beating the path: Sample-efficient Monte-Carlo planningNeural Information Processing Systems, Dec 2016, Barcelona, Spain |
|
|
Online learning with noisy side observationsInternational Conference on Artificial Intelligence and Statistics, May 2016, Seville, Spain |
|
|
Pliable rejection samplingInternational Conference on Machine Learning, Jun 2016, New York City, United States |
|
|
Analysis of Nyström method with sequential ridge leverage score samplingUncertainty in Artificial Intelligence, Jun 2016, New York City, United States |
|
|
Rewards and errors in multi-arm bandits for interactive educationChallenges in Machine Learning: Gaming and Education workshop at Neural Information Processing Systems, 2016, Barcelona, Spain |
|
|
Revealing graph bandits for maximizing local influenceInternational Conference on Artificial Intelligence and Statistics, May 2016, Seville, Spain |
|
|
Pack only the essentials: Adaptive dictionary learning for kernel ridge regressionAdaptive and Scalable Nonparametric Methods in Machine Learning at Neural Information Processing Systems, 2016, Barcelona, Spain |
|
|
Online learning with Erdős-Rényi side-observation graphsUncertainty in Artificial Intelligence, Jun 2016, New York City, United States |
|
|
Black-box optimization of noisy functions with unknown smoothnessNeural Information Processing Systems, 2015, Montréal, Canada |
|
|
Large-scale semi-supervised learning with online spectral graph sparsificationResource-Efficient Machine Learning workshop at International Conference on Machine Learning, Jul 2015, Lille, France |
|
|
Cheap BanditsInternational Conference on Machine Learning, 2015, Lille, France |
|
|
Maximum Entropy Semi-Supervised Inverse Reinforcement LearningInternational Joint Conference on Artificial Intelligence, Jul 2015, Bueons Aires, Argentina |
|
|
Simple regret for infinitely many armed banditsInternational Conference on Machine Learning, Jul 2015, Lille, France |
|
|
Spectral Bandits for Smooth Graph FunctionsInternational Conference on Machine Learning, May 2014, Beijing, China |
|
|
Spectral Bandits for Smooth Graph Functions with Applications in Recommender SystemsAAAI Workshop on Sequential Decision-Making with Big Data, Jul 2014, Québec City, Canada |
|
|
Efficient learning by implicit exploration in bandit problems with side observationsNeural Information Processing Systems, Dec 2014, Montréal, Canada |
|
|
Extreme banditsNeural Information Processing Systems, Dec 2014, Montréal, Canada |
|
|
MESSI: Maximum Entropy Semi-Supervised Inverse Reinforcement LearningNIPS Workshop on Novel Trends and Applications in Reinforcement Learning, 2014, Montreal, Canada |
|
|
Spectral Thompson SamplingAAAI Conference on Artificial Intelligence, Jul 2014, Québec City, Canada |
|
|
Online combinatorial optimization with stochastic decision sets and adversarial lossesNeural Information Processing Systems, Dec 2014, Montréal, Canada |
|
|
Bandits attack function optimizationIEEE Congress on Evolutionary Computation, Jul 2014, Beijing, China |
|
|
Learning from a Single Labeled Face and a Stream of Unlabeled Data10th IEEE International Conference on Automatic Face and Gesture Recognition, Apr 2013, Shanghai, China |
|
|
Finite-Time Analysis of Kernelised Contextual BanditsUncertainty in Artificial Intelligence, Jul 2013, Bellevue, United States |
|
|
Stochastic Simultaneous Optimistic OptimizationInternational Conference on Machine Learning, Jun 2013, Atlanta, United States |
|
|
Semi-Supervised Apprenticeship LearningThe 10th European Workshop on Reinforcement Learning (EWRL 2012), Jun 2012, Edinburgh, United Kingdom. pp.131-141 |
|
|
Conditional Anomaly Detection Using Soft Harmonic Functions: An Application to Clinical AlertingThe 28th International Conference on Machine Learning Workshop on Machine Learning for Global Challenges, Jun 2011, Seattle, United States |
|
|
Conditional Anomaly Detection with Soft Harmonic FunctionsProceedings of the 2011 IEEE International Conference on Data Mining, Dec 2011, Vancouver, Canada |
|
|
Feature importance analysis for patient management decisions13th International Congress on Medical Informatics MEDINFO 2010, Sep 2010, Cape Town, South Africa. pp.861-865, ⟨10.3233/978-1-60750-588-4-861⟩ |
|
|
Semi-Supervised Learning with Max-Margin Graph CutsInternational Conference on Artificial Intelligence and Statistics, May 2010, Chia Laguna, Sardinia, Italy |
|
|
Online Semi-Supervised Learning on Quantized GraphsUncertainty in Artificial Intelligence, Jun 2010, Catalina Island, United States |
|
|
Online Semi-Supervised Perception: Real-Time Learning without Explicit Feedback4th IEEE Online Learning for Computer Vision Workshop, Jun 2010, San Francisco, United States. ⟨10.1109/CVPRW.2010.5543877⟩ |
|
|
Conditional anomaly detection methods for patient-management alert systemsWorkshop on Machine Learning in Health Care Applications in The 25th International Conference on Machine Learning, Jul 2008, Helsinki, Finland |
|
|
Learning predictive models for combinations of heterogeneous proteomic data sourcesAMIA Summit on Translational Bioinformatics, Mar 2008, San Francisco, United States |
|
|
Distance Metric Learning for Conditional Anomaly DetectionTwenty-First International Florida Artificial Intelligence Research Society Conference, May 2008, Coconut Grove, Florida, United States |
|
|
Evidence-based Anomaly Detection in Clinical DomainsAnnual American Medical Informatics Association Symposium, 2007, Chicago, United States. pp.319--324 |
|
|
A Comparison of Chief Complaints and Emergency Department Reports for Identifying Patients with Acute Lower Respiratory SyndromeInternational Society for Disease Surveillance, Oct 2006, Baltimore, United States |
|
|
Evolutionary Feature Selection for Spiking Neural Network Pattern ClassifiersProceedings of 2005 Portuguese Conference on Artificial Intelligence, Dec 2005, Covilha, Portugal. pp.181-187, ⟨10.1109/EPIA.2005.341291⟩ |
|
|
Game plan: What AI can do for football, and what football can do for AIJournal of Artificial Intelligence Research, 2021, 71, pp.41-88. ⟨10.1613/jair.1.12505⟩ |
|
|
Fast sampling from beta-ensemblesStatistics and Computing, 2021, 31 (7), ⟨10.1007/s11222-020-09984-0⟩ |
|
|
Spectral banditsJournal of Machine Learning Research, 2020 |
|
|
DPPy: Sampling Determinantal Point Processes with PythonJournal of Machine Learning Research, 2019 |
|
|
Bayesian Policy Gradient and Actor-Critic AlgorithmsJournal of Machine Learning Research, 2016, 17 (66), pp.1-53 |
|
|
Outlier detection for patient monitoring and alertingJournal of Biomedical Informatics, 2013, 46, pp.47-55. ⟨10.1016/j.jbi.2012.08.004⟩ |
|
|
Identification of microbial and proteomic biomarkers in early childhood cariesInternational Journal of Dentistry, 2011, 2011, pp.196721. ⟨10.1155/2011/196721⟩ |
|
|
Conditional Outlier Detection for Clinical AlertingAMIA .. Annual Symposium proceedings [electronic resource] / AMIA Symposium. AMIA Symposium., 2010, 2010, pp.286-90 |
|
|
Bandits on graphs and structuresMachine Learning [stat.ML]. École normale supérieure de Cachan - ENS Cachan, 2016 |
|
|
Adaptive Graph-Based Algorithms for Conditional Anomaly Detection and Semi-Supervised LearningOther Statistics [stat.ML]. University of Pittsburgh, 2011. English. ⟨NNT : ⟩ |
|
|
Feature Selection and Dimensionality Reduction in Genomics and ProteomicsWerner Dubitzky, Martin Granzow and Daniel Berrar. Fundamentals of Data Mining in Genomics and Proteomics, Springer, pp.149-172, 2006, ⟨10.1007/978-0-387-47509-7⟩ |
|
|
Evolving Neural Networks for Statistical Decision TheoryMachine Learning [stat.ML]. 2005 |