There is already a petition with c.50 votes to remove the admin colour scheme selector from the profile page. But it also has mixed comments, with a number of users saying it should remain.
I suspect the main problem is the prominence on the page it is given, because you always have to scroll past it to get to the more useful information.
This petition is for a simple fix that moves the ‘Personal options’ section down the page. It can still be used and the options changed, but it isn’t the first thing you see.
I moved the whole table section in
user-edit.php starting with
<h2 class="user-profile-personal-options"><?php _e( 'Personal Options' ); ?></h2>
below the ‘Account management’ table. I am not sure if this will cause any broader issues, but it works fine for me on a test site.
I can put in a PR for this, if people think it worth doing.