On paper, if you calculate the numbers, I'm almost 100% certain that roads full of self driving cars would be a lot safer than the current roads full of human controlled cars. The issue is that people like to have a sense of control, and also that some of us (myself included) quite like driving.
Also, you can already commercially get cruise control systems that maintain a certain distance to the car in front (sorting out the acceleration/braking for you).