In order to compare continuing with Mind Sear to stopping for a Mind Blast, we need to figure out exactly how much damage Mind Blast is doing for us. It hits hard, sure, but it also gives us a shadow orb, so we'll add in the value of an orb, 1/3 of a DP, and also add the value we'd get from Insanity from that 1/3 DP (if we're talented for it, which we almost always are currently). Then, see how many targets it would take for Mind Sear to be better dps than stopping to Mind Blast. Although in T16 we use Insanity for most of the fights, I'm going to break this down separately for when you do and do not have Insanity talented. Insanity adds extra value to shadow orbs, and since the value of shadow orbs is important to what we're looking at, this distinction is necessary.
Here's the tricky thing, though. This assumes you're missing an entire Mind Blast and all of the extra damage that comes with it by continuing to Mind Sear. Are we really? If you Mind Sear for 5 seconds after MB comes off cooldown, Mind Blast is still there to use afterwards. What we did was delay it by 6/9ths (5s late + 1s cast time) of its usual interval (8s cd + 1s cast time = 9s). So really we lost 6/9ths of a Mind Blast and everything associated with it. To take the delay of Mind Blast rather than complete loss of it into account, we need to multiply the total MB damage by (t+1)/9, where t is the length of time you expect to be Mind Searing while MB is ready to be used.
With Insanity Talented:
Mind Sear:This damage begins at 315 + 0.3*SP every 1 sec for 5 sec. Multiply that by 1.5 because a Mind Blast takes 1.5s to cast (both scale with haste, so if you're like me and have a 0.66s tick interval for Mind Sear, you also have a 1.0s Mind Blast cast time, which is again a 1:1.5 ratio) and we get the..
Total for Mind Sear: (472.5 + .45*SP)*numTargets, where numTargets is the number of targets being hit by Mind Sear
Now we'll take MB, 1/3 of a DP assuming you use it at 3 orbs, and 3 ticks of Insanity's increase in damage to a Mind Flay (assuming we'd get 9 ticks of Insanity from the DP, 3 ticks is 1/3 of that Insanity bonus) to get the total damage that actually results from a MB. Subtract 1/2 a Mind Flay tick to compensate for 1/3 of a GCD that you spend using the orb you get from this Mind Blast, too.
Mind Blast: 2768.5 + 1.909*SP
1/3 of a DP @ 3 orbs: 3286 + 1.572*SP, but haste gives us extra ticks at break points, so we need to add those in. At and beyond 14,873 haste, we have +3 ticks of DP. Many of us are at that point, so I'll assume this number and add in 3*(274 + 0.131*SP) for 3 more ticks of 1/3 of a 3 orb DP, which gives us a total of 4109 + 1.965*SP.
3 ticks of Insanity bonus: 3147 + 1.5*SP (note that this is only the extra damage to Mind Flay added by Insanity, since if we didn't DP we'd be doing regular Mind Flay, so we subtract that damage out to get the increase given from the Mind Blast's orb)
-1/2 of a Mind Flay tick: -524.5 - 0.25*SP
Total from Mind Blast: (mbDelay+1/9)*(9500 + 5.12*SP), where mbDelay is the time using Mind Blast will be delayed after it's off cooldown.
Since we're interested in the difference between these two values, we simply subtract Mind Sear's damage from Mind Blast's damage. You'll notice that SP doesn't cancel out, which means Spell Power matters. You'll also notice that Mind Blast's SP multiplier is quite a bit higher until we have 12 targets to Mind Sear, so the more spell power you have the stronger MB is compared to Mind Sear until you reach 12 targets. In order to reduce a variable for now, I'm going to be looking at 50,000 Spell Power. If you don't have that much or have even more, don't worry. It doesn't change that much, but feel free to calculate it out for your own levels if you're way below 50k. :)
So now we're factoring in the number of adds and how long we're going to be delaying Mind Blast. This feels like graph time to me. To make your own, simply set the above two totals equal and then solve for numTargets or mbDelay, then plug in relevant possibilities for those variables and fill a table with your results.
Well, there it is. You'll want a rough estimate of how many seconds you'll be delaying Mind Blast. Then, get a good idea of how many things you will be Mind Searing. A good take-away from this is that if there are 11 targets (think Immerseus AoE), you can just continue happily Mind Searing forever. If there are fewer than that, say 5, and you think they'll be around for a while, then keep Mind Blast on cooldown. If they'll be dying pretty quickly (3s or less), go ahead and delay Mind Blast to burst them down really quickly. As another example, on Garrosh phase 1, we get 7 targets (Garrosh + 6 melee adds). The first set dies very quickly in my guild, so I won't be interrupting for Mind Blast. The later sets tend to last a bit more than 4.5 seconds, so I'm better off interrupting MS for a quick MB.
Without Insanity:
This part is much easier to figure out. We're simply comparing Mind Sear to Mind Blast plus 1/3 of a DP minus 1/3 of a GCD of Mind Flay filler.
From the above, nothing changes for Mind Sear.
Total for Mind Sear: (472.5 + .45*SP)*numTargets, where numTargets is the number of targets being hit by Mind Sear
On the other side, we'll take Mind Blast, add in 1/3 of a 3-orb DP, and subtract out 1/3 of a GCD for that DP. We'll again use Mind Flay filler for that GCD.
Mind Blast: 2768.5 + 1.909*SP
1/3 of a DP @ 3 orbs, +3 ticks haste break point: 4109 + 1.965*SP
-1/2 of a Mind Flay tick: -524.5 - 0.25*SP
Total for Mind Blast: (mbDelay+1/9)*(6353 + 3.62*SP), where mbDelay is the time Mind Blast will be delay after it's off cooldown.
And of course we're going to look at the difference between these two things in relation to both the number of targets and the time you expect to be delaying Mind Blast, so it's time for another graph! This one also assumes 50,000 spell power.
So, if you're not talented for Insanity, delaying Mind Blast becomes a better option more often. You can optimally spam Mind Sear forever and ignore orbs if there are 8 targets that will be hit by it. Not having the extra damage from Insanity baked into Mind Blast makes a noticeable difference!
Limitations:
First, this post hasn't mentioned procs. Going into detail to cover them would make this post insanely long. Briefly, procs tend to favor Mind Searing. Much of the damage baked into Mind Blast comes later on in the form of DP and Insanity, and they likely won't be benefiting from the procs. I also assumed the +3 DP haste break point. If you don't have that much haste, then the total MB damage is overvalued for you slightly. Most people should have at least +2, though, which isn't hugely different. I also assumed 50,000 Spell Power. Spell Power definitely affects the out, so if you have 20k SP, you'll want to recalculate things. The difference isn't drastic for anywhere near but not exactly at 50k SP, though. If you're in mostly T16 normal or better gear, this is easily close enough. Also, if you're spec'd for DI, the interval calculation isn't always going to be accurate. You'd lose a lot more dps by having a DI proc get refreshed by another one during Mind Sear, though, so go ahead and use procs during AoE. Finally, this post assumes you'll actually use Insanity well and aren't orb capped. If you won't be able to use Insanity because there's lots of stuff to Mind Sear, and Sear will be better than Insanity due to the number of targets, then treat it as though you don't have Insanity talented.
I hope that will help clear things up when it comes to an aspect of AoE that has not been fully covered in guides before! Please feel free to send me questions about this or any other aspect of playing a Shadow Priest!