Oh, I don’t think it was that simple.
One of the may convenient pegs for people to hang their hat was that — agreed.
But the UK has (had?) a great deal with the EU, and merely failed to even explain that regularly to its people.
Instead we rely upon peddlers of misrepresentation and opportunists like David Cameron and George Osbourne to set the scene. And that was never going to end well.