Staring with the basics, potatoes will take about 12 weeks from planting to cropping, and the plants will be killed by the first frost. (The actual potatoes will be fine since they are underground, but the plant's leaves won't survive).
I don't know what the climate is like where you live, but midsummer is probably about the latest time you can usefully plant outside. On the other hand, you can grow potatoes in pots or containers indoors, if you have a large enough space and plenty of light. A single potato plant will grow leaves that cover an area more than a foot in diameter. Indoors, you can plant them in August and harvest "new potatoes" at Christmas.
"Grocery store potatoes" are not the best source of seed, because you probably don't know what variety they are (i.e. whether they are "early" or "late" varieties - quick or slow growing). They may also have been in storage for a long time, and treated with a growth inhibitor. They may also have been infected by plant diseases which have no effect on humans, but may affect growing them for a second season. "Genuine" seed potatoes are grown separately from potatoes intended for eating, and special precautions are taken against disease infection.
If the potatoes are carrying virus diseases, they may infect the soil and the diseases may be transmitted to related plant species in subsequent years. The most common related plants in a vegetable garden are tomatoes, aubergines, and chillies, but there are also common garden flowers like nicotiana (tobacco plant) and petunias.
Just a final safety warning since you mentioned a 3-year-old: your potato plants may flower, and then set seeds which look like small tomatoes. The flowers can look quite pretty, but the seeds are poisonous. Don't forget that potatoes are related to deadly nightshade! Removing the seeds and disposing of them safely won't affect the potato crop.