Quick followup… just in case some else find the same bug.
I followed the instructions here [N] French accented letters are not correctly displayed - Desktop editors - ONLYOFFICE Community
Deactivate “graphics acceleration”, restarted OnlyOffice. Nothing changed. Turned all cells to Bold (nothing changed), turned back to regular : went back to normal.
Thanks to the teams on the other thread for the info !