Add the frozen peaches, agave nectar (or honey), yogurt and lemon juice to the bowl of a food processor. Process until creamy, about 5 minutes.
Serve the frozen yogurt immediately or transfer it to an airtight container and store it in the freezer.