Crops are expected to be down by as much as 50% in places due to a combination of heavy rain and little sunshine.
Producers have said it has been one of the most challenging seasons in living memory.
The warning came as a major conference looking at the latest developments in potato farming opened in Dundee.
Some 700 farmers have gathered at the Potatoes in Practice conference at Balruddery Farm. The event is Britain's largest field-based potato industry event.
Challenging season
The site is used by the James Hutton Institute to put research into practice, and gives farmers the chance to see the latest developments first hand.
Balruddery manager Ewan Cauldwell said: "It's been fairly bad, we've faced the same challenges as any of the other growers in the area.
"Right from planting it's been very wet, we've had a lack of sunshine, there's been a lack of temperature, the disease issues have been particularly bad, the pressures on disease have been bad, the ground conditions have been wet and the opportunity to get on control these diseases has been challenging in itself."
He said the season had been challenging for other crops as well not just potatoes, but the main concern for growers was just how bad harvests were going to be.
Drought focus
He warned: "I think we've probably lost yield already. I think we won't get that back, particularly if there are areas of the fields that have been waterlogged and there's been a lot of that."
Finlay Dale, a potato breeder at the James Hutton Institute, said despite the recent wet summers, the long term focus was on crops that are able to resist water shortages.
He said: "The big issues are not so much this year, which is very challenging for growers. The real problems are blight control, which are long term problems, trying to reduce chemical input into the crop, drought and water management.
"Water is going to be expensive - not this year - but it is going to be a real cost."
Varying conditions
Mike Storey from the UK Potato Council said the picture was not completely bleak.
He said: "Around the country, you've got a range of conditions and the supply chain groups are working together with the growers to make sure the supermarkets and the processors get the crops they need.
"It's too early to say what the conditions are going to be like right across the country at the end of the harvest, but the growers are working together with their supply chains to ensure the consumers get what they need."
