DataRec Library for Reproducible in Recommend Systems

Summary of DataRec Library for Reproducible in Recommend Systems

by Kyle Polich

32m•November 13, 2025
Listen to Episode

Overview of DataRec Library for Reproducible in Recommender Systems

This episode of Data Skeptic (host Kyle Polich) interviews Alberto Carlo Maria Mancino about a new open‑source Python library—referred to in the conversation as DataRec (sometimes heard as “DataRack/DataReq” in the transcript). The library aims to make offline recommender‑system research more reproducible and less error‑prone by standardizing dataset retrieval, preprocessing (filtering/splitting), versioning, and traceability so researchers can focus on models and evaluation rather than ad‑hoc dataset plumbing.

Key topics covered

  • Motivation: common reproducibility problems in recommender‑system research (ad‑hoc dataset copies, hidden preprocessing changes, non‑traceable experiments).
  • Typical recommender dataset workflows and preprocessing needs (sparsity, filtering users/items, temporal splits).
  • What DataRec offers: canonical dataset sources, checksums, unified readers/objects, exportable transformation traces (YAML), and compatibility with existing frameworks.
  • Practical adoption: how researchers can use the library, current maturity, and roadmap (tutorial at RecSys 2025; documentation/examples forthcoming).
  • Best practices and advice for reproducibility (checksums, seed fixing, traceable configs).

Background: recommender research & dataset pain points

  • Most academic recommender work is done via offline evaluation on public datasets (e.g., MovieLens, Last.fm, Amazon review collections, Gowalla).
  • Datasets are often sparse (each user has few interactions vs. huge item catalogs), so preprocessing is common: filtering users/items with low interaction counts, temporal filtering/splitting, rating transformations, etc.
  • Small, untracked changes (different versions of the same file, different split methods, platform differences) can materially change reported results and make comparisons difficult or impossible to reproduce.

What DataRec does (features & workflow)

  • Central idea: provide a lightweight, plug‑and‑play Python library for dataset management rather than a monolithic experiment framework.
  • Canonical dataset retrieval: DataRec points to original dataset sources (reconstructs dataset provenance) so users download the correct canonical files rather than random copies.
  • Checksum verification: downloaded files are checksum‑verified to detect changes to source files.
  • Unified data object & readers: regardless of file format (CSV/TSV/JSON), DataRec wraps datasets in a consistent DataRec object and provides dataset‑specific readers so the same pipeline code can work across datasets.
  • Preprocessing utilities: built‑in filtering, splitting (including temporal splits), and other common transformations used in recommender research.
  • Traceability/export: every transformation can be traced and exported as a YAML configuration file. That YAML can be used to reproduce the same processed dataset.
  • Export/interop: exports are designed so DataRec outputs can be fed into existing reproducibility frameworks and pipelines (the library aims to complement—not replace—these frameworks).

Practical deliverables and usage

  • Raw files are downloaded as published (JSON, CSV, etc.), but DataRec converts/wraps them into a standard in‑memory object with readers.
  • You can run identical preprocessing across different datasets without restructuring your code—just swap the DataRec dataset object.
  • Export includes YAML config + checksum metadata so others can reproduce the exact same dataset transformations and final processed files.
  • Installation & status: planned to be published on PyPI; at the time of the interview it was newly released on GitHub (July) and in early stages. Examples and documentation were expected to be published before RecSys 2025.

Reproducibility advice from the interview

  • Trace transformations: record every major preprocessing step (filtering, splitting, seed values).
  • Use checksums to fingerprint datasets—especially at the beginning and end of your preprocessing chain (computing checksums at every step can be expensive).
  • Fix random seeds and document configurations; export a human‑readable config (YAML) so others can rerun the exact pipeline.
  • Community view: the recommender community cares about reproducibility (e.g., reproducibility track at RecSys); improved tooling speeds research and benchmarking.

Use cases / who benefits

  • Researchers who need a standard way to fetch canonical datasets and apply consistent preprocessing across experiments.
  • People wanting to re‑run published experiments and verify results without hunting down exact dataset versions or manual data cleaning scripts.
  • Early‑stage researchers who want fast prototyping without the overhead of full framework integration.
  • Framework developers who want to delegate dataset management to a dedicated library and focus on modeling and evaluation.

Roadmap & adoption

  • Early stage: library published recently (July), adoption just starting. Tutorial planned at RecSys 2025 to promote usage.
  • Short‑term goals: publish documentation and usage examples, stabilize the package for PyPI.
  • Mid‑term goals: integrate DataRec into popular recommender frameworks (example contact with CoreNAC authors); ship versions pre‑integrated into frameworks to standardize the data‑management layer.

Actionable steps (how to get started)

  • Check the project repository and examples (project GitHub) and read the usage examples once docs are published.
  • Expect to:
    • Install the package (pip once published to PyPI) or clone the GitHub repo while it’s in early release.
    • Use a DataRec dataset object in your scripts so you can apply standard readers, filters, and splits.
    • Export YAML transformation configs and include checksum metadata when publishing results.
  • If you run into issues, contact the maintainers/authors (they encouraged users to reach out).

Notable quotes

  • “Let’s try to make at least this part standard…so everyone agrees on how we should download and link to these datasets.”
  • “If we don’t do research properly…we are just slowing down the research.”

Where to follow / more info

  • Look for the DataRec (DataRack/DataReq in parts of the transcript) GitHub repo and examples; documentation was imminent around RecSys 2025.
  • Alberto Mancino: reachable via Twitter and his personal webpage (search his name online).

Summary takeaway: DataRec is a lightweight, dataset‑centric Python library built to standardize dataset provenance, preprocessing, and versioning for offline recommender‑system research. It focuses on traceability (checksums, YAML export) and interoperability with existing frameworks—addressing common reproducibility pitfalls so researchers can compare models more reliably and iterate faster.