Hello again!
This suggestion in fact already exists! There are some toilets near Kings Cross in unmagical London, where you can click on levers above the toilets to be transported to the Ministry. We encourage you to explore a bit to find them!
Again, thank you for suggesting this and have a wonderful day!