Alignment

Over time WOW dungeons have changed greatly.  In the beginning dungeons were often brutal mazes, designed for people to wander in and fight for hours without really knowing where they were going.  When entering a dungeon with a particular goal in mind you definitely needed an experienced leader to make sure you went the right way and in some cases like Blackrock Depths even people who had been there a dozen times wouldn't know it all.  Recently I saw a complaint about current dungeons that sums up the changes to WOW dungeons nicely:

The real trouble with designing dungeons like the bottom one here is that while it is great for those hunting for lore or exploring for fun it is a disaster for those looking to do one particular thing.  It requires a knowledgeable guide, getting to any particular event is going to require a ton of random killing of monsters, and even if you know where to go there is no guarantee your companions will have any interest in doing the same things you do.

The fundamental conflict is that optimizers will be in a dungeon to do one thing.  Explorers will be there to do any number of things.  If those groups are in a complex mazelike dungeon they will drive each other crazy - either the optimizers will have to take a slow, meandering route to get where they want (or might not even get to where they want at all!) or the explorers will only get to see a single path to the final boss.  There is no way to keep everyone happy.  In a simple dungeon the explorers get to see all the things and the optimizers get to rush to the goal as fast as possible - the optimal paths for both groups are the same.

That, I think, is how Blizzard arrived at their current design.  They really want people to be able to play with random strangers and have a good time and the only way to do that is to make sure that all the major groups of players are going to want to take the same path.  Doing so requires that the path choices be nonexistent though, which certainly cuts down on some of the fun of the thing.  They can't really make both kinds of dungeons though, as we know what happens then:  Everyone goes to the fastest, simplest, quickest route to rewards and runs it over and over while complaining that the big, complex dungeon is too annoying.  All the dungeons have to have vaguely similar complexity or only the trivial ones will be used, and if they all have similar complexity then they all need to be simple or the casual players will just give up on them.

I don't think Blizzard employees actually feel like the linear dungeons are the best possible or most fun design.  Hell, I bet they wish they could design outrageous mazes like Blackrock Depths.  That would be so much fun!  However, the players have made it clear that the designs of old will not do.