Erdogan, Turkey's most popular modern leader but also its most divisive, had hoped a crushing victory for the AKP would allow it to change the constitution and create a more powerful U.S.-style presidency. To do that, it would have needed to win two-thirds of the seats in parliament.

Instead, it has been left unable to govern alone for the first time since it came to power almost 13 years ago. It faces potentially weeks of difficult coalition negotiations with reluctant opposition parties as it tries to form a stable government, and the possibility of another early election.

With 98 percent of ballots counted, the AKP took 40.8 percent of the vote, according to broadcaster CNN Turk, down from 49.8 percent at the last parliamentary election in 2011.

"Everyone should see that the AKP is the winner and leader of these elections," a defiant Prime Minister Ahmet Davutoglu, leader of the AKP, said in a balcony speech to the party faithful at its headquarters in Ankara.

"No one should try to build a victory from an election they lost," he told thousands of supporters.

The uncertainty sent the lira currency to a record low against the dollar in thin out-of-hours dealing as investors positioned themselves for what is likely to be a turbulent start of trade on Monday.

But for jubilant Kurds, who flooded the streets of the southeastern city of Diyarbakir setting off fireworks and waving flags, there was plenty to celebrate. The pro-Kurdish Peoples' Democratic Party (HDP) crossed a 10 percent threshold to enter parliament for the first time.

With initial results putting it on around 13 percent, HDP co-leader Selahattin Demirtas ruled out a coalition with the AKP and said the election outcome had put an end to talk of the stronger presidential powers championed by Erdogan.

"The discussion of an executive presidency and dictatorship have come to an end in Turkey," he told a news conference in Istanbul, describing the outcome as a victory "for those who want a pluralist and civil new constitution".

The AKP's failure to win an overall majority marks an end to more than a decade of stable single-party rule and is a setback for both Erdogan and Davutoglu.

Both men had portrayed the election as a choice between a "new Turkey" and a return to a history marked by short-lived coalition governments, economic instability and coups by a military whose influence Erdogan has now reined in.

"Erdogan is the main loser given that he championed two big ideas: one a switch to a presidential system, the other single-party government," said Sinan Ulgen, visiting scholar at Carnegie Europe and chairman of the Istanbul-based Center for Economics and Foreign Policy Studies.

"Neither of them came about."

EARLY ELECTION SEEN

The partial results indicated that the HDP, with its roots in Kurdish nationalism, had succeeded in widening its appeal beyond its Kurdish core vote to center-left and secularist elements disillusioned with Erdogan.

It is now likely to play a significant role in parliament, particularly trying to advance a two-year-old peace process between the Turkish state and the Kurdistan Workers Party (PKK) militant group, which first took up arms in 1984.

Demirtas said earlier that the campaign had not been fair or just. A bombing on Friday killed two people and wounded at least 200 at one of its rallies in Diyarbakir.

The results broadcast by CNN showed the secularist Republican People's Party (CHP) would again be the second biggest group in parliament, with around a quarter of the vote.

Murat Karayalcin, the party's Istanbul chairman, said the outcome was a "clear no" to the executive presidential system championed by Erdogan, while party spokesman Haluk Koc ruled out a coalition with the AKP.

The right-wing Nationalist Movement Party (MHP), long seen as the AKP's most likely partner in any coalition, took around 16 percent of the vote.

Its leader, Devlet Bahceli, all but ruled out a deal with the AKP, saying Turkey should hold a new election if the ruling party was unable to agree a coalition with other opposition parties.

"The first possibility ... should be between AKP and HDP. The second model can consist of AKP, CHP and HDP," he said. "If all these scenarios fail, then early elections must be held."

A senior AKP official also said a coalition with the MHP was unlikely, and that the ruling party would rather go it alone and try to build support back up ahead of a new, early election.

"If there is an AKP-MHP coalition, then we will not be able to achieve even this level of votes at the next election," the official said.

The lira has fallen more than 15 percent against the dollar this year on uncertainty about the election outcome, making it one of the worst-performing emerging market currencies.

Many investors had hoped for a weakened single-party AKP government, avoiding the uncertainty of a coalition but without handing Erdogan greater power.