[ENH] Centralize parallel backend selection in skbase and introduce typed config class

[SimonBlanke]

Is your feature request related to a problem? Please describe.

There is no parallel backend API in skbase today, while sktime, skpro, and Hyperactive each carry their own backend selection and configuration using the pair backend and backend_params.

Describe the solution you'd like

I propose to add a small skbase.parallel module that defines a common interface for parallel execution. Once skbase provides this shared primitive, downstream projects can remove duplicate code and depend on the central API. This is a rough sketch how those classes could look like:

class LokyParallel:
    n_jobs: Optional[int] = None
    prefer: Optional[str] = None 
    require: Optional[str] = None 

class RayParallel:
    ray_remote_args: Optional[Dict[str, Any]] = None
    shutdown_ray: bool = True
    logger_name: str = "ray"
    mute_warnings: bool = False

class DaskParallel:
    scheduler: Optional[str] = None 

Describe alternatives you've considered

A nested dictionary with a single top level key was considered instead of a dataclass because it encodes the dependency in one parameter. This is harder to explain and invites deep untyped structures, which burden documentation and error messages.

[fkiraly]

I really like this idea!

I have some questions about the design, to be resolved we implement.

Regarding the plan, I would carry this out in two stages:

1. add the "backends" framework in scikit-base
2. point the "backends" framework in sktime, skpro, hyperactive to scikit-base
3. add the dataclass / config class based interface, as an alternative means to configure

[fkiraly]

Regarding the design: I am not 100% happy that these classes have to be imported, and more backends require more classes to be added, making the extension locus for new backends more complicated

Alternatives:

• flat dictionary? With key "backend" additionally to backend specific keys. Unfortunately some backends have the backend key, for secondary backends (like joblib)
• a single config class, not sure how this would exactly look like