Asian, Latino, and Middle Eastern markets are excellent for low-cost, naturally GF items like rice noodles, lentils, chickpeas, and masa harina.
Planning weekly meals around what is already in your pantry prevents impulse buys. Cooking in large batches and freezing portions reduces the temptation to buy expensive GF "ready meals".
Many standard products (e.g., certain brands of corn chips or pasta sauces) are naturally gluten-free but not labeled as such. Checking standard ingredient lists can save you from paying for specialized GF certification.