Conflict is they key to an interesting plot. It doesn't necessarily need to be violent conflict. It could be as simple as getting information. The hero wants some information so he goes to a known street rat to get it. If the street rat gives it up immediately then there is no conflict. In other words it is boring and could easily have been narrated away without the player's input. It's more interesting if the street rat has reasons to not give it up, but to make the hero earn it. Maybe the street rat is afraid of being found out as a snitch, maybe he is greedy and must be bartered with for the information, or maybe he just sees no reason to help the heroes until they earn his fear or respect.
Here's another example: The heroes are sneaking through the villain's manor when one of the party guests wanders into the room where they are hiding. The guest isn't actively searching for the heroes but his mere presence can make their heart skip a beat.
When you improvise conflict, just remember this simple formula. The heroes want something, but then something gets in the way of that goal. That's conflict.
It could just be something as simple as the heroes want to search the room for treasure, but then there is an irate Minotaur in the way. That is the conflict of most early D&D dungeon modules (aka dungeon crawls), and in some cases that is enough.
This may read like it's all very basic stuff, but the panic of improvising can make you forget some very basic things.
So as your players drift from your pre-planned story line and begin to explore a different path, you must be ready to introduce Conflict.
Finesse
Now it's time to talk about the art and the skills you will need so that you don't frustrate your players with the Conflicts you introduce.While it's okay to throw a seemingly impossible task in front of the players every now and then that forces them to really think, it's bad form to do this for every Conflict. The more difficult you make a task, the more time it will take for the players to figure out a way through it. Too many of these tasks will bog down the game and your players will get bored. Try to keep some level of fairness in mind with your challenges with respect to the game world. The challenges should be solvable by your heroes, otherwise the game cannot continue.
One GM I played with ran a Star Wars game where we as rebel mercenaries had to get our assignment from a covert operative on a space station. Weapons were not allowed on the station so we had to go in unarmed to get our assignment. The conflict our GM threw at us was that the contact was crazy (why then would the rebellion trust or use him as a middle-man?), he had an explosive vest strapped to his chest (on a station where weapons were strictly forbidden), and he wouldn't give us our assignment unless we paid him an exorbitant amount of money (that we didn't have).
So paying him was not an option, reasoning with him was not an option, and threatening him was not an option. We did what any mercenary would do in that circumstance. We said screw the money and we left. Game over.
It is the job of the GM to challenge the players, not to defeat them. After all you control the final ruling, the setting, and the infinite number of villains. What sort of challenge is it for you to beat the players when your are god.
Twists and Turns
Next lets talk about spicing up your conflict. Think about things that can add a twist to your conflict. Let's say the heroes want the info from that street rat. I gave you a bunch of examples of standard motivations and solutions but what about something not so standard.What if the street rat is desperate to free his girl from a drug den so they can run off and make a fresh start, but he lacks the strength and firepower to rescue her? So he has a price for the heroes, but his motive isn't greed.
Let's say the heroes are entering a cavern to slay monsters and get treasure. Here's a twist: The monsters are organized and have built defensive fortifications.
And Then the Ninjas Attacked!
Sometimes the heroes may get confused and not know where to go. Or the players might be getting bored with too much character acting or puzzle solving. If the pace of your game starts to drag, it's time to send in the ninjas. BTW it's always good to have a generic "Thug/Ninja" npc statted out for your game in case your need one...or a dozen.
Throwing an action scene at the players should wake them up. Basically the villain has had enough of the heroes asking questions and sends a death squad to "take care of them" (I hate that cliched line). And oops, their just happens to be a clue on one of the death squad members that can lead the party back on track.
This is a good trick to keep in your repertoire. The players will be so happy that something is happening that it's unlikely they will complain about the obvious tactic.
Summary
So when you have to improvise a conflict, remember to make it a challenge but not an impossible task, especially if the plot cannot move forward unless the heroes deal with this conflict.Make it fit with the setting and what is going on. Always leave a path for the heroes to succeed. And given that this is improv, if your players come up with a cool idea to deal with the situation that you hadn't thought of, go with it.