Unfortunately, you need to get it all off, as the plastic primer will react/curdle with the yellow primer, If you have some thinners at hand, it may be easier to soak a rag, and wipe the yellow primer off (No smoking etc, and well ventilated area, and HD gloves) once nice and clean, then you can apply the Plastic primer, followed by your Hi build primer (Do NOT sand any of the primers between coats) 6ish coats at this stage should be sufficient, let dry over night, any rough areas can the be lightly sanded to remove the nibbs, any areas that rub through, should be re primed, but try and avoid rubbing through if you can...
Then prep for paint, as in my post
Hope this helps
Thom