Significant minority? Rather more than the odd isolated case, unusual individual or group but by no means half or more of the workforce.

In my previous public sector job I would have put it at perhaps 20% that were either unwilling or unable to perform their jobs effectively.