If those contract manufacturers are making something you do not produce under your SQF facilities, then you could say they're outside of scope. But that would mean you shouldn't warehouse them or contribute to their creation at all. An auditor would love to ding you for the appearance that you're letting someone make your SQF certified products elsewhere lol. Something as simple as showing those products on your company website while elsewhere stating you are SQF certified could be murky enough to present a finding (for potential misuse of their shield logo).
Just food for thought. If those products aren't being represented as an SQF product, then you have the leeway to exclude those contract makers and those products from the scope of your audit.