The preschool and early elementary years are a wonderful time to homeschool. Young children take delight in so many things and can really be a joy to teach. The young years also seem to require a lot of stuff, from educational tools to art supplies. My oldest is starting middle school studies and I can attest to the fact that his supply needs are much smaller now than they were during kindergarten.
Early education doesn't require a textbook or even a dedicated learning space, but you do need to organize. Math activities, phonics games, science tools and a variety of music, art and learning supplies can quickly overtake your home. If you have a dedicated learning space, great! If not, you need to at least have a dedicated storage space.
If cost is a factor, hit the thrift stores. We used an attractive $20 armoire in the dining room to house our school supplies when we first began homeschooling my oldest. Our home was the size of a two-bedroom apartment, measuring no more than 900 square feet, so space was at a premium. I had dollar store baskets on the shelves, each holding specific subject materials and labeled prominently on the front so we could only pull out what we needed. The rule was one basket had to be put away before we pulled out the next!
I had a file system for daily work, such as worksheets. A plastic five-drawer organizer sat inside the armoire. Each drawer was labeled with a day. On Sunday evenings I would place all the papers in the correct drawer. This gave my son some self-guided work he could do when I busy with his baby brother or if I was sick and couldn't “do” school that day.
After we moved into a larger home and I began schooling two boys, we were able to dedicate a whole room to school. The skills I learned about organizing early on continue to pay off. Our school room is organized and a pleasure to work in to this day, and we haven't given in to the temptation to fill it up with too much stuff.