The Hashd al-Shaabi militia has faced accusations of abuses against civilians in Sunni-majority areas.
Speaking to supporters on Friday, al-Sadr called for dissolving the militia and merging its fighters in the Iraqi army.
"The Hashd al-Shaabi…is for Iraq and will not be dissolved," al-Abadi said in a conference in the capital Baghdad.
"The next phase after liberating the land from Daesh is the battle of the unity of word," he said.
The Hashd al-Shaabi militia was established in 2014 with the avowed purpose of fighting the Daesh terrorist group, which captured vast swathes of territory in northern and western Iraq.
The Shia militia, however, has faced accusations of abuses against civilians in Sunni-majority areas.
Last month, the Iraqi army recaptured Mosul, the country's second largest city, from Daesh, which overran the city in 2014.