Weirdly enough, I don't equate defective with physically handicapped -- and sorry for the ableist language there.

Defective seems to be used for the people who have mental issues ranging from severe depression to schizophrenia. The underlying theme seems to be, "if they'd just pull themselves together they could be useful members of society."