F factor

I've got a 20 something Cast Hunter and I've notice he's got a dark reddish black cloud of what i assume is F-factor. Does anyone know how it happened and how to get rid of it?

I assume you are using the Negative Photon Back Unit on this character? If so, you just need to click on the Unit in your Inventory and select the Hide (or is it Stealth on the NA version) option since it is just the Unit's visual effect.

F-Factor build up is not an actual gameplay mechanic, it is just part of the story.