Nuxeo item in Calisphere: URL to PDF in S3 is indexed by Google (not item landing page) / update robots.txt?

[aturner]

Reported by Lisa at UCSF: a Google search for one of their items results in a link to the source PDF file (and not the Calisphere item landing page): https://www.google.com/search?q=Care+Through+Parents%253A+Creating+Space+in+Pediatric

Do we need to update our robots.txt exclusion rule, to minimize indexing of the downloadable links?

• CSphere item: https://calisphere.org/item/b897e788-8662-464b-944b-562c6c7a2098/
• Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/b897e788-8662-464b-944b-562c6c7a2098/view_documents

Freshdesk
https://help.oac.cdlib.org/a/tickets/175382