spelling standardised checkers.

[wolfiex]

Issue Title

No response

Description

A find and replace, regex expression function to standardise descriptions,
e.g 'ModeLLing' or cmip -> CMIP

Context

No response

Acceptance Criteria

• Task 1
• Task 2
• Task 3

Additional Information

No response

[durack1]

This pretty horrible nested code to validate UTF-8 encoding worked great - could be completely rewritten far more gracefully, but it basically sanitized all entries so that a standard character set, and a standard type was imposed on all CVs - it basically used the string.decode("utf-8") and string.encode("utf-8") pairs to remap special EU chars (umlauts etc) into their standard ascii character matches. For good measure there is a py2 -> py3 check in there, as the CMIP6 CVs evolved through that transition.

https://github.com/WCRP-CMIP/CMIP6_CVs/blob/006b6e88397d74f4aee97e4f43a8db833f605c10/src/writeJson.py#L1330-L1394

[durack1]

The other ones mentioned this am are

US spelling	King's English spelling
aluminum	aluminium
analyze	analyse
center	centre
color	colour
license	licence
modeling	modelling
paleo	palaeo (not always)
program	programme (not always)
sulfate	sulphate

Lots of online resources, this one was interesting https://www.scribbr.com/category/us-vs-uk/

@eleanororourke @sashakames @sol1105 @ltroussellier @glevava @taylor13 ping

[eleanororourke]

I think sulphate/sulfate is other way around @durack1 ? US is the sulfate and UK is the sulphate. Program/me is a bit nuanced in UK - generally a computer program but a programme of events.

We better be careful not to get sucked into this too much!

[durack1]

I think sulphate/sulfate is other way around @durack1 ? US is the sulfate and UK is the sulphate.

Yep, you're right! Fixed above, iteration in action.. And agreed, this is an off to the side task, but will start to become a larger issue as the QA/QC steps start to solidify