I personally get annoyed when that quest is failed as it means people have essentially waisted boosters along with time. As for the whole abandoning the quest thing. I've only seen it a couple times and both were situations were 10+ minutes into the fight the total damage done to him wasn't enough to get him to start using his time attacks where he changes the lighting then does attacks either faster or slower than normal.
Having LV50's isn't so much of an issue on its own. However when its lv 50's that aren't geared for it to where they're atleast using an element he's weak to it gets to be an issue that grows worse the more people like that are present. Pair that with a number of them that are regularly decorating the floor because they don't attempt to avoid attacks and it just starts snowballing. I've even seen groups fail the pillars DPS check on the minions mission before that on SH.