Dichotomous Forecasts for Monthly Run

[marinamatic]

Implement Dichotomous Forecasts in the New Pipeline

Description:

The dichotomous forecasts, which are currently derived from ensemble point predictions rather than a separate model, have not yet been implemented in the new pipeline. These forecasts are essential for users of the API and dashboard, where they are expected as part of the regular monthly delivery. Failing to implement them would disrupt service and break dashboard functionality that relies on these outputs.

This issue focuses on integrating the generation of dichotomous forecasts into the updated pipeline to ensure continuity and prevent disruptions for downstream users.

Key Areas to Cover:

• Review Legacy Implementation: Understand how dichotomous forecasts were previously generated and delivered. The concerned code (for cm) can be found here, starting from the section "Retrieve ensemble predictions for test partition to create categorical predictions". The pgm counterpart can be found here:
• Integration Strategy: Define how to reproduce this logic in the new pipeline using existing ensemble point predictions, OR build a new model producing dichotomous ensemble predictions (we do not need dichotomous output for constituent models).
• Pipeline Placement: Determine where in the new pipeline the dichotomous forecast generation step should occur.
• Validation: Ensure output matches expected format and structure required by the API and dashboard.

Objective:

To implement dichotomous forecasts in the new pipeline, ensuring uninterrupted delivery to the API and dashboard consumers.

Tasks:

• Review the method for generating dichotomous forecasts from emseble point predictions in the old pipeline.
• Implement equivalent logic in the new pipeline, or build new model.
• Ensure the outputs are compatible with the API and dashboard delivery expectations.
• Test the integration and validate output with historical deliveries.

[angelicalmcgowan]

@jimdale I added reference to building a new dichotomous model in the issue above following our discussion just now. Please add/edit as needed.

@Polichinel @marinamatic FYI: Issue description updated to include possible options of developing a new dichotomous model (ensemble only – we do not need dichotomous output from constituent models).