To add an element of surprise and to show what your players can expect in the future, add a mechanic or unique situation to your boss the players haven’t seen yet. This can be something as simple as regeneration, or as complex as making it so they can only defeat it under certain circumstances.
Rather than just throwing in a random mechanic, choose or create something that relates to the boss itself, the story, or a trait that relates to an even greater antagonist that the players may or may not be aware of.
Here are three boss battles that add special mechanics to the fight.
The Obsidian Forge
Your players enter your bosses’ lair: an active forge built into a volcano that makes obsidian tools and weapons. Pipes carrying lava snake in and out of the walls of the cave. As they make their way deeper inside through winding caves, they find the boss (who is immune to fire) near flowing lava. The workers, scared of the PCs, flee. With no one working the forge, the machinery starts to fail. If the players do nothing, the pipes will burst, filling the caves with lava. Thus, the players must defeat the boss while making tweaks to the forge machinery to keep it from trapping and killing them.
The tweaks the players need to make to the forge could be minor or major. You could make it obvious what levers to pull, or make players make some sort of check to figure out how it works and how to stop the pipes from bursting. All of that depends on the pace of the fight, the style of your game, and what you as a GM like keeping track of.
The key to survival is all of the players working together to fight the boss and keep the machinery running in tandem. To keep the smart characters from only working the forge and the tough and strong players from fighting the boss the whole time, mix up what they need to do to the forge.
For example: after a player passes a check and learns how the forge works, have the task require a feat of strength greater than what they can do. Have a stuck lever or jammed valve require a stronger member of the party to break away from the fight and perform the task, which may send the smarter character to fight the boss for a turn or two.
Don’t just have players use their turn throwing a switch. Make it more dynamic and meaningful to the story. One idea is having the sweltering heat from the troubled machines cause players to take a penalty on their attack rolls. The player working the forge could resolve the issue, decreasing or eliminating the penalty. The player working the forge could also find ways to use the environment in their favor, such as dousing players to give them a bit of temporary protection from the heat, or dousing the boss if they think it will harm it.
The Possessed Boss
I love the regenerate head effect that Hydra’s have (Pathfinder reference here: http://www.d20pfsrd.com/bestiary/monster-listings/magical-beasts/hydra). The ability to have their heads cut off only to have two grow out of the stump is nothing short of badass. However, I almost never see this kind of ability used on any other creature.
Have a creature possessed by a powerful group of outsiders (demons are a great choice for this, but you can choose whatever fits your game). When the boss has a limb struck with a piercing or slashing weapon and it does a certain amount of damage, the limb is cut off but two demonic limbs replace it. The boss gains an additional attack action with the new limb. Each limb can do an unarmed attack, use a one-handed weapon, or two limbs can hold a two-handed weapon together for extra damage.
If the boss’ head is struck with enough force to sever it, take a lesson from the hydra and have two heads grow in its place. If the creature can cast spells (using verbal components), has a breath weapon, or has a bite attack, give the boss an extra action to use it.
Rather than the only win condition for the boss being bringing its HP to zero, the players can defeat the boss by hitting specific parts of it. If the creature is humanoid, have the targets on the torso, like the lower of the five chakra points on the human body. If the boss is mechanical or magical in nature, give it some lightly-masked, external weak points they can exploit to shut it down. If the boss is a monstrosity, just do whatever you want because monsters can have whatever anatomy you can imagine. Regardless of your choice, the specific spots should be harder to hit than the normal AC or defense of the boss.
If you want to take another lesson from the hydra, make it possible to seal the stumps so nothing can grow out of them. For example, if something demonic possesses the boss, the players could douse the stump in holy water before the new limbs/heads/whatevers grow out of it, sealing the wound so nothing can spring from it.
The Goopy Wizard
Split is one of my favorite abilities (Pathfinder/d20 reference here: http://www.d20pfsrd.com/bestiary/monster-listings/oozes/pudding-black). The basic idea that you can use with any system is this: when a creature is struck with a piercing or slashing weapon, instead of taking damage, the creature splits into two copies of itself. Each copy has half of the creature’s prior total HP but all of its abilities, equipment, etc.
Here’s how it can work for a boss battle:
Work a spellcaster antagonist into the campaign. The caster doesn’t have to be especially menacing or connected directly to a critical part of the story. It can be a leader of a local gang, a mercenary, or a lackey for your main antagonist. The main point is to let the players learn that they’ll need to fight a powerful spellcaster. As far as they know, he is a powerful but fairly normal spellcaster.
Allow the players to ambush or face the caster when it is alone. This makes it more enticing for them to cut him down as quickly as possible.
However, when they strike the caster with a sharp weapon, it splits into two. This allows each caster to cast spells on their turn, making the fight about twice as difficult as the players thought it would be.
But don’t just go for a normal split effect. Give it some additional flavor. Two additional things happen every time the caster splits.
First, the two parts don’t just fall to either side, they explode outward, sending each caster far off (30-60 feet) in opposite directions. This keeps casters from being easily surrounded and adds space between them and the party so the casters have more time and room to cast spells.
Second, the explosion coats all nearby creatures (about 15 feet) with goop from the explosion. The goop sticks to the characters and their clothing/armor. The goop quickly hardens, lowering the affected creatures’/players’ speed by around one-third (round it depending on what system you are using). This makes it more difficult for them to get to the casters, giving the casters another advantage.
There’s a lot you can do by slipping in some additional mechanics and environmental changes. I didn’t even cover adding story points to a boss fight. I’ll have to save that for another post.