You are not way off ... you are right to be thinking about this.
As your clay surface picks up contaminants, continually fold/knead the clay to expose a clean surface. After a certain number of uses, even this process won't allow you to create a clean clay surface and you should delegate that clay for the trash can, or maybe for glass or wheels.
If you ever drop a piece of clay bar, do not try to salvage it for use as you'll never get out all the grit.