Informfully Recommenders

Welcome to Informfully (GitHub & Website)! Informfully is an open-source reproducibility platform for content distribution and user experiments.

To view the full documentation, please visit Informfully at Read the Docs. It is the combined documentation for all code repositories.

Links and Resources: GitHub | Website | X | Documentation | DDIS@UZH | Google Play | App Store

Pipeline Overview

Informfully Recommenders is a norm-aware extension of Cornac. Please see the Experiments Repository for an overview of our past offline and online studies using this framework as back end. And see the Online Tutorial for a quick introduction on how to use this repository. Furthermore, we provide sample recommendations of past experiments for testing and development purposes.

Please find below an overview of the norm-aware Extension of datasets, models, re-rankers, and metrics for which Informfully Recommenders provide out-of-the-box support. Please note that this repository is fully backward compatible. It includes and supports all elements that were already part of Cornac.

Pre-processing Stage

Dataset	Language and Type	Links
EB-NeRD	Danish News Dataset	Website
MIND	English (US) News Dataset	Website
NeMig	German News Dataset	Website

Augmentation	Links
Sentiment Analysis	Script
Named Entities	Script
Political Actors	Script
Text Complexity	Script
Story Cluster	Script
Article Category	Script

Splitting	Link
Attribute-based Sorting	Link
Diversity-based Subset Construction	Link
Attribute-based Stratified Splitting	Link
Diversity-based Stratified Splitting	Link
Clustering-based Stratified Splitting	Link

In-processing Stage

Diversity Algorithms	Description	Links
PLD	Participatory Diversity	Paper, Code
EPD	Deliberative Diversity	Paper, Code

Random Walks	Description	Links
D-RDW	Diversity-Driven Random Walks	Paper, Code
RP3-β	Random Walks	Paper, Code
RWE-D	Random Walks with Erasure	Paper, Code

Neural Model	Description	Links
EMNF	Neural Baseline	Paper, Code
LSTUR	Neural Baseline	Paper, Code
NPA	Neural Baseline	Paper, Code
NRMS	Neural Baseline	Paper, Code
VAE	Neural Baseline	Paper, Code

Post-processing Stage

Re-ranker	Description	Links
G-KL	Greedy Kullback-Leibler Divergence	Paper, Code
PM-2	Diversity by Proportionality	Paper, Code
MMR	Maximal Marginal Relevance	Paper, Code
DAP	Dynamic Attribute Penalization	Paper, Code

Simulator	Links
Rank-based User Simulator	Paper, Script
Preference-based User Simulator	Paper, Script

Evaluation Stage

Metric	Description	Links
Gini	Gini Coefficinet	Paper, Code
ILD	Intra-list Distance	Paper, Code
RADio	RADio Divergence	Paper, Code

Gini Dimension	Required Augmentation
Category Gini	Article Category
Sentiment Gini	Sentiment Analysis
Party Gini	Political Actors

ILD Dimension	Required Augmentation
Category ILD	Article Category
Sentiment ILD	Sentiment Analysis
Party ILD	Political Actors

RADio Dimension	Required Augmentation
Activation	Sentiment Analysis
Calibration	Article Category, Text Complexity
Fragmentation	Story Cluster
Alternative Voices	Political Actors, Named Entities
Representation	Political Actors, Named Entities

Sample Scripts	Links
Accuracy Evaluation (AUC)	Script
Traditional Diversity Evaluation (Gini and ILD)	Script
Normative Diversity Evaluation (RADio)	Script

Item visualization is done using the Informfully Platform. Please look at the relevant documentation page for a demo script.

Citation

If you use any code or data from this repository in a scientific publication, we ask you to cite the following papers:

• Informfully Recommenders – Reproducibility Framework for Diversity-aware Intra-session Recommendations, Heitz et al., Proceedings of the 19th ACM Conference on Recommender Systems, 2025.

  @inproceedings{heitz2025recommenders,
    title={Informfully Recommenders – Reproducibility Framework for Diversity-aware Intra-session Recommendations},
    author={Heitz, Lucien and Li, Runze and Inel, Oana and Bernstein, Abraham},
    booktitle={Proceedings of the 19th ACM Conference on Recommender Systems},
    pages={792--801},
    year={2025},
    publisher={ACM New York, NY, USA},
    url={https://doi.org/10.1145/3705328.3748148}
  }

• Informfully - Research Platform for Reproducible User Studies, Heitz et al., Proceedings of the 18th ACM Conference on Recommender Systems, 2024.

  @inproceedings{heitz2024informfully,
    title={Informfully - Research Platform for Reproducible User Studies},
    author={Heitz, Lucien and Croci, Julian A and Sachdeva, Madhav and Bernstein, Abraham},
    booktitle={Proceedings of the 18th ACM Conference on Recommender Systems},
    pages={660--669},
    year={2024},
    publisher={ACM New York, NY, USA},
    url={https://doi.org/10.1145/3640457.3688066}
  }

• Multi-Modal Recommender Systems: Hands-On Exploration, Truong et al., Proceedings of the 15th ACM Conference on Recommender Systems, 2021.

  @inproceedings{truong2021multi,
    title={Multi-modal recommender systems: Hands-on exploration},
    author={Truong, Quoc-Tuan and Salah, Aghiles and Lauw, Hady},
    booktitle={Proceedings of the 15th ACM Conference on Recommender Systems},
    pages={834--837},
    year={2021},
    publisher={ACM New York, NY, USA},
    url={https://doi.org/10.1145/3460231.3473324}
  }
  

Contributing

You are welcome to contribute to the Informfully ecosystem and become a part of our community. Feel free to:

• Fork any of the Informfully repositories.
• Suggest new features in Future Release.
• Make changes and create pull requests.

Please post your feature requests and bug reports in our GitHub issues section.

License

Released under the Apache License 2.0. (Please note that the respective copyright licenses of third-party libraries and dependencies apply.)