Is this where a 51% attack could make this plausible? 1 person broadcasting a block to the network would immediately be rejected but if 5100/10000(for example) broadcast the same fabricated block, this could fool the system, correct?
No. Legitimate nodes will never accept a block that is considered invalid, regardless of how many nodes are broadcasting it. In your example, the attacking nodes would form a separate fork and the legit nodes would just carry on with the original blockchain.
The first reponse and this one pretty much cover it. You would be on a chain only with those set up similarly to yours