Dr Andrew McLellan also said continued overcrowding was the single biggest threat to cutting reoffending.
The HM Prisons Inspectorate for Scotland report calls for the issue to be "defeated" as a matter of urgency.
Justice Minister Cathy Jamieson said there was no "short-term fix" but that investment, rehabilitation and community sentencing were all key.
Dr McLellan also condemned the practice of slopping out and said cost-cutting was hitting staff levels.
The impact of the best strategies in the best prisons carried out by the best staff is hopelessly weakened by overcrowding
Dr Andrew McLellan
Chief inspector of prisons
The former Moderator of the General Assembly of the Church of Scotland said overcrowding had worsened in the past year and that the increasing population was having far-reaching effects.
Dr McLellan said: "The facilities which provide the services to the prison do not grow larger when the prison becomes overcrowded."
He said prisons should focus on preventing inmates from reoffending when they were released, but acknowledged overcrowding in some jails made this "almost impossible".
Dr McLellan said: "To make Scotland safer, prison overcrowding must be attacked and defeated.
Slopping out was branded "humiliating"
"The impact of the best strategies in the best prisons carried out by the best staff is hopelessly weakened by overcrowding."
In a landmark Court of Session ruling earlier this year, Lord Bonomy found slopping out amounted to degrading treatment under the European Convention on Human Rights.
Dr McLellan highlighted the case and said progress had been made in the last 12 months to end slopping out.
But he cautioned: "Until the number is reduced to none we cannot call our prisons decent."
Citing the example of Polmont young offender's institution near Falkirk, Dr McLellan hit out at remand prisoners having to take part in the practice, attacking "the humiliation and squalor".
The prisons watchdog described the relationship between inmates and staff as good, but warned it could be endangered by overcrowding.
But he did welcome investment in prison buildings, despite admitting he initially thought the money could have been better spent.
He concluded the only way to reduce re-offending was to reduce offending and called for the development of initiatives aimed at young people.
Responding to the report, the justice minister said a modernisation programme to tackle conditions and overcrowding had to be accompanied by wider ranging sentencing and rehabilitation measures.
Ms Jamieson said: "We need solutions that extend beyond the prison walls.
She added: "The use of electronic monitoring for low risk offenders and drug treatment and testing orders also offer effective diversions from jail.
"These are not 'soft' options but stringent measures that allow offenders to reorient and reintegrate themselves."
Who cares. You do the crime you do the time and quite frankly they deserve nothing more than over crowding and slopping out.
Edited by Lottie, 06 October 2004 - 10:12 AM.