From the photo you provided, this appears to be brown rot, a very common fungal disease affecting peaches.

What causes brown rot

Brown rot is a fungal disease that thrives in warm, humid conditions and spreads by spores. It commonly infects blossoms, shoots and fruit, then overwinters in mummified fruit left on the tree or ground, ready to re-infect the tree the following season.

How to help prevent brown rot

• Good garden hygiene is very important. Remove all shrivelled or mummified peaches from the tree and the ground at the end of the season, as these can carry the disease through winter and into the next season. 
• After pruning, also remove and dispose of all pruned material, along with any fallen leaves left on the ground after early winter leaf fall. This helps reduce the number of fungal spores that can infect the next crop.
• Winter pruning is important. Prune to create good airflow through the tree and allow sunlight to reach the branches, as brown rot is more likely to develop where growth is dense and humid conditions linger. 
• A preventative spray can also help. Applying copper oxychloride before blossom in spring is commonly recommended to help control brown rot infection.

Brown rot tends to be more severe in warm, humid climates, so maintaining good airflow, good hygiene and a preventative spray programme will give your peach tree the best chance of producing healthy fruit.

Unfortunately, as climate changes continue, some parts of New Zealand that have traditionally grown peaches well may become less suitable, as brown rot thrives in hot, humid conditions.