Right in the corner of the yard is a nice location for annuals or perennial plants, but not a great location for a tree. Try to imagine the tree in 10 years time, it will want to spread out and will overhang quite a bit the other side of the fence. Roots will spread in a circle, running under the fence to who knows what on the other side. We can see for example some construction in one neighbouring property. Have you investigated what the tubular chimney like things are attached to, and what might be entering the ground as a result? What might neighbours do on their property that might involve cutting back the roots of your tree?
The yellowing in one sense is natural - the tree is removing nutrient from those leaves to feed new growth and likely those yellow leaves will fall naturally and if the tree can find enough nutrient from the roots then it is not a big problem. However the tree can only rely on one quadrant of four directions to spread, so it will inevitably run into problems at some point.
The tree is still young and might be moved with care to a better location. Have you considered moving it to the middle of your yard where it would have room to grow and be productive? This might conflict with other uses, but at some point in the future you might revisit this idea when the tree is much larger and more of a problem to handle.