After completing the final shrine and getting the 4th spirit stone, the old man will visit you and say that you need to imagine an X with the ends being the shrines that you just visited. He will meet you at the point where they intersect. In the adventure log this shows up as being "the place centered amid the food shrines". Where exactly is it that I meet this guy?
It took me some thinking to figure out what he actually meant, but once I got it, it made sense. To make a long story short, you need to head to the temple of time. Upon arrival, you will notice a statue at the end of the church that is lighting up. Head over here and interact with it. After this is complete the old man will show up.
If you are having trouble getting onto the roof, head around to the front of the church and climb up on the broken wall. This will allow you to get up without needing a large amount of stamina.
