This is also something that I find to be quite annoying.
Even with the tap square solution aside, the "avoid the pan on every right click" reasoning doesn't apply when middle clicking, so having a delay is even more unnecessary in that scenario.
panning view delay
Re: panning view delay
Agreed! I am still here and suffering!