When I first picked up the D&D 4e rulebooks and began designing my campaign, my DMG was opened several times to the "Encounter Level" chart in order to determine what my XP budget was, and I was using that as a basis for all the encounters I put together. After all, this was the sort of thing severely lacking in previous editions, and Wizards of the Coast must have gone through the motions of balancing both sides to a conflict... They must know what they're doing to come up with these numbers... right?
As the first group in the campaign I'm currently running approaches their fourth level and the end of the first act, I look at MasterPlan and see every other box is in bright red, as if MasterPlan is telling me "are you insane?!? This encounter will crush the party in to gibs! TPK! TPK! For god's sake, man... TPK!!!"
Why is that? Because the current party, when presented an encounter equal or even one level higher than their own, plows through them as if they weren't even there. An army of minions? No problem! A dozen or more kobolds/goblins/small nuisances? Piece of cake! A solo monster two levels higher than the rest of them? Child's play! It was brutal, but not for the players. Even the "hard" encounters didn't last more than two or three rounds, and it usually ended up with only one player or two injured. It feels like they're never bloodied.
I thought something was wrong... this wasn't the way it was supposed to be, right? Granted, some of the problems were actual design issues (my solo "boss" wasn't properly designed, for example), but that couldn't have been the case always. "Maybe it was the dice's fault... Yeah, that's it..." (admittedly, there were some really bad die rolls on behalf of the monsters), but to compensate I found myself adding monsters, traps, or something more to the mix to make it feel like more of a challenge. For example, going on the mathematics alone, at least one non-boss encounter ended up being five levels higher than the party.
The second group to run the campaign had a little more difficulty. The encounters were virtually the same, but they weren't getting off so easily. We recently finished an encounter that would qualify as "hard" (in terms of the XP allowance) and almost every party member (even two NPC allies) got bloodied. I fear that if I throw the same Level+5 encounter at them, it will be a soul crushing defeat.
At the table you can adapt to this; if the party is having it easy, throw some more at them. If the party is having trouble, you can throw some allies in to the mix, or take some monsters away, or even fudge the dice in the player's favor. But I'm designing a campaign for physical distribution... I don't have that luxury, do I?
So I'm forced to create my encounters using the formula the DMG provides because those numbers theoretically define what the expected difficulty should be for an "average" party. In the back of my mind I can't help but think the encounter is too easy, but I can't beef the encounter up at will because I have no idea the ability of the party facing it. An encounter I consider "easy" could be devastating when thrown against a party with a different makeup or a different level of experience.
I can't blame the dice. To me, dice in a DM's hand are sometimes optional... the DM can ultimately overrule them anyway, so he could technically decide hits and misses based on what best fits the story (NOTE: Whenever I have overruled my own dice rolls, it has always been in favor of the party; I don't make it worse for the players just because I feel like it). If the DM wants the party to squeak out of the encounter by their fingernails, he could easily do that without rolling a single die. Sure the players get to roll on their own, and the DM has no say over those results, but he could most certainly compensate by downgrading a monster attack roll or two.
I realize now that one of the important aspects in module design isn't necessarily difficulty but entertainment. Unless I'm making something "fourthcore", I kind of have to go by the recommendations because they are the norm. If the end result is a pushover for the party, or if the party is getting hammered to the brink of death, I have to have a certain degree of trust in the DM running the campaign to make up for that.
As part of my campaign, I've considered adding a section to each encounter or scene describing how to make it "harder" or "easier". I've seen some modules describe how to adjust the difficulty in cases where there are more or less players ("if four players, do this..." "If six, do this..."), but the ones I've seen have provided very general recommendations at the beginning of the module, not on a per encounter basis. And most modules don't discuss the topic at all, expecting the DM to figure that sort of thing out as he goes. The way I see it, I either have faith in the DMs to compensate or release two versions: a seemingly wimpy (at least to me), by-the-book module and a Fourthcore "no, seriously, everyone's gonna die" version.
I guess it all boils down to playtesting. I'm currently running the campaign in three groups, but I do admit they are all "play by post"; I have not tried any part of my campaign with a live audience. One of these days I should run it in person, but I haven't DM-ed a live game in almost two decades.
Also, before I DM one of my own, I feel I need to play a lot more. A lot of people find that somewhat puzzling... It was the same problem when I was doing video game design, writing The Opera (total conversion for Half-Life). When I told people "I don't have time to play games, I'm too busy writing them!" they thought I was joking, but it's the honest truth. Hopefully I'll remedy this concern soon.
Time will tell.