I've only had success with retail purchased items when we can quantify that it is for R&D purposes. I see a conflict in buying retail items in supplier approval:
2.3.4.2 requires you to setup a supplier approval program and requires you to maintain 7 key points in regards to these suppliers. I don't know how you get agreed specifications, a summary of food safety controls from the supplier, and a manner in which to review supplier performance when you're merely buying off the shelf retail items. The concern would be that your supplier, aka the retailer, can't guarantee the things you buy won't be discontinued (which puts business continuity into question on your end), nor that they won't change vendors and leave you with a product you haven't validated for your process.
2.3.4.3. requires you to verify raw materials with COA's, or sampling and testing. Again, not sure how you get that retail.
Now, 2.3.4.4. would allow you to get raw materials from non-approved suppliers, but only in an emergency situation and would be subject to a receiving inspection or analysis. Showing that you're doing this routinely would be a hit against this code.
I've worked for companies using the "we're a small business and need to do xyz to make things work" and it can get you through audits with an understanding auditor. If I were in your shoes, I'd try to shift as much of your raw material purchases to brokers you can approve as suppliers, where possible. If you get rejected by brokers due to low volume, keep those rejections handy and use them to support your need to buy the items retail as an emergency (your need to continue production but inability to acquire from approved suppliers may qualify under emergency purchases). Make a list of the items you have to buy retail, and send a letter to your certifying body before the audit asking for notice of exemption for supplier approval on those items. That way the CB knows you have a round peg that won't fit their square hold ahead of time, and you'll have had a chance to explain how you justify the risk.