Blackboxes are not loaded from hidden packages that are explicitly exposed with `-package`, `-package-id`

[pieter-bos]

Clash can find modules the same way that GHC can using all the normal flags, including the ability to hide / expose previously exposed / hidden packages with e.g. -package and -package-id. From a cursory look it seems that the -package-dbs are passed along to hint for loading blackboxes, but not -package and -package-id. It would be nice if we can use the same set of visible packages for loading top modules and blackboxes.

Our use case here is that it is nice to have testing only HITL top entities neatly stashed away in an internal library, rather than having to expose them through the main published library. When building the library through nix, there is no option at present to build an internal library to be exposed. Nevertheless building the main library builds all the components, just with the internal libraries (rightly) hidden:

$ /nix/store/w4..m7-ghc-9.10.3-with-packages/bin/ghc-pkg list
/nix/store/w4..m7-ghc-9.10.3-with-packages/lib/ghc-9.10.3/lib/package.conf.d
(...)
    clash-crypto-0.1
    (z-clash-crypto-z-hitlt-instances-0.1)
    (z-clash-crypto-z-hitlt-shared-0.1)
    (z-clash-crypto-z-shake-rules-0.1)

Note that e.g. z-clash-crypto-z-hitlt-instances is the munged name of clash-crypto:hitlt-instances, an internal library of clash-crypto. Specifying -package-id (...hitlt-instances...) works fine to load the top entity from that module, but any blackboxes cannot be loaded from that packge with this database state.

Plenty of workarounds are there, but it would be nice if we can avoid them. Nevertheless some options that are around are:

• Just unhide the package: ghc-pkg expose z-clash-crypto-z-hitlt-instances
• Use a separate cabal file for all testing components, with all the required modules in the main library.