Race condition where the consumer exists but no username and database

[vincenzodnp]

Today we had an edge case because the manager container of the dbaas operator pod was in OOM.
For a project, we had the mariadbconsumer object created and the spec.consumer info correctly filled in, but actually, no username, password, and database were created in the cluster.

Checking the code, it seems the creation of the username (password) and database happens AFTER the consumer's Infos are defined into the k8s object.

[shreddedbacon]

The database / user creation happens here https://github.com/amazeeio/dbaas-operator/blob/main/controllers/mariadb/mariadbconsumer_controller.go#L171

And the update of the k8s resource happens here https://github.com/amazeeio/dbaas-operator/blob/main/controllers/mariadb/mariadbconsumer_controller.go#L238

So the username and database are created prior to the k8s resource being updated.

If the MariaDBConsumer resource had the username, password, and database, along with all the associated services created, but there was no matching database or user in the actual destination MariaDBProvider, then that would be interesting to see.

You don't happen to have the logs from the operator when this was occuring? It would have been vomiting up errors the entire time it tried to do anything.