Eliminating distributors would be a death blow to small manufacturer and retailers. Retailers would need to be part of larger chains that can share some of the administrative and logistics costs. Most producers are already owned by some sort of conglomerate, but it would become impossible for any startups not backed by one of them. Retailers and producers would both need to be owned by larger companies that have corporate offices that essentially function as distributors. Now the power is even more consolidated among a select few, and you end up with monopolies. There could be some benefits, but history tells us this is usually bad for the consumer.
Your statement "its not all clicks on the internet nowadays!", conflicts with your idea of eliminating distributors. Quick and easy one click ordering is the only way small manufacturers could hope to make any sales. Even though every retail store has hired an additional person to place orders, they still wont have time to actually call each manufacturer. Everything would have to be highly automated.
I am all for for a dual channel system, but eliminating distributors would cause costs to skyrocket and selection and availability to plummet. That would be a sad day for enthusiasts. It would probably drive me to stop drinking.