Student assistant (40 hours per month)

The DEEM Lab (https://deem.berlin) conducts research at the intersection of data engineering and machine learning. Our goal is to develop efficient data systems for AI and machine learning applications that are easy to use while simultaneously guaranteeing the fundamental rights of users (such as the "right to be forgotten").

We are looking for a Student Research Assistant to support the development of a new benchmark for evaluating machine learning engineering (MLE) agents with more realistic ML pipelines. Current benchmarks rely heavily on Kaggle-style tasks that are likely contaminated in LLM training data, limiting their validity, and quite trivial to solve, limiting the representativeness. This project aims to create a more realistic evaluation framework with diverse, non-contaminated tasks spanning tabular and multi-modal data and evaluating not only performance but also quality, validity, and novelty of the MLE agent created code.

Tasks to be completed under supervision:

• Dataset collection and curation (40%)
• designing and implementing evaluation tasks covering prediction, data integration, fairness, and robustness (30%)
• evaluating MLE agent performance and analyzing results (30%)

The student assistant will be included as a co-author on resulting scientific publications.

Required Qualifications:

• Solid knowledge of machine learning (concepts, models, and workflows)
• Proficiency in implementing machine learning pipelines in Python, with experience using libraries such as scikit-learn, pandas, polars, transformers, or PyTorch
• Prior experience with data science competitions (Kaggle), benchmark design, dataset curation, or contributions to ML frameworks
• Good knowledge of German and/or English required; willingness to acquire the respective missing language skills

Nice to Have:

• Experience with MLE agent frameworks (AIDE, SWE) or MLE benchmark suites (MLEBench, MLE-Dojo)
• Prior exposure to open-source development practices or contributions to open-source projects

Party responsible for specialist area / point of contact for job posting: Prof. Dr. Sebastian Schelter - sekr@deem.tu-berlin.de
Period of employment: immediately for 2 years
Apply to: sekr@deem.tu-berlin.de

Please submit your written application including cover letter, your CV, certificate of enrollment, and where applicable, current transcript of records, with the reference number to the place of employment indicated above.
In the interest of promoting equality opportunities for men and women, applications from women with suitable qualifications are particularly encouraged.