I've made this analogy before, this is a bit like asking "do not build an access ramp in my apartments building unless needed", this will automatically prevent anyone with disabilities to move into that building, so there you go... problem solved right?
Someone said, "well, if it is my house, I should be able to chose", the reason that rationale doesn't work is because the GNOME Shell experience should be designed to be inclusive, so this is really closer to an office building or an apartment building instead of a private house.
We could add an option in GSettings and let it be switchable from gnome-tweak-tool, but that's about it. Removing the accessibility button from my point of view is wrong. If the accessibility button is not accessible... how are people with disabilities supposed to access it?