That's partly because I don't enjoy driving. However, a lot of people do. Many of those people promise that they will never buy a self-driving vehicle. I propose that laziness will drive that promise right out of them.
Today, even people who own cars will occasionally take a taxi. To the airport, or out drinking, or when traveling. As taxis become autonomous, they will be even more convenient. Imagine tapping your smartphone, then 30 seconds later a car arrives for you, and you can step inside and keep dicking around on your phone, or have a meal, or get work done, until it drops you off right at your destination. And it only costs a few dollars.
Even people who love driving will take advantage of that once in a while. At first maybe it'll only be to get to the airport. But then it'll be when they have a deadline coming up, or are really hung over, or are just feeling lazy.
As those situations become more common, and driving your own car becomes less common, the per-trip cost of owning a car becomes prohibitive. Is it worth tens of thousands of dollars in purchase price, fuel, maintenance, and insurance just to drive a car once a day? Once a week? What about once a month?
"I'm too lazy to drive, just this once" can quickly become "I haven't driven in a month and I might as well sell my car." As more and more people succumb to laziness and rely on a cloud of autonomous vehicles, houses will gradually lose their driveways and garages, and the thrill of driving will be confined to go-kart tracks.
In short, human laziness will lead to a more efficient, car-ownership-free world.
I think it'll be a good change. The people who disagree will be too lazy to resist it.