Divorce advice needed, but complicated

So we got married in mauritius a few years ago. Marriage is legally recognised in UK although we never registered it afterwards. It's not a requirement although you can.

Now we are getting divorced due to adultery on his part.

I never actually changed my name on anything such as passport, bank accounts etc. As far anything financial or legal I'm still in my previous name and never been married. Do I have to divorce?

I'm hoping not because the Mauritian government did a weird thing with our marriage certificate. They put it in my birth name. Over the years I had changed my name by deed poll in early 20's and I had also been married prior. I go by my first married name as never changed it back after my first divorce, but my 2nd marriage certificate is in my birth name and apart from a birth certificate I have nothing else in that name. They insist on marriage certs bring in your birth name.
Hope. This makes sense.

We have no children and nothing financial to sort. It would be a clean break

You do need a divorce, yes. You are married because you were legally married in Mauritius. It doesn't matter that the marriage was not registered in the UK. You are still married and therefore you need a divorce if you want to dissolve the marriage.

It doesn't matter that the name on the marriage certificate and the name you go by are different. As long as the marriage certificate refers to you that is all that counts. This type of situation is not that uncommon. It just means that on the divorce petition the position has to be explained quite carefully and accurately.

Thank you