Tens of thousands gathered for peace ceremonies in Hiroshima on Tuesday to mark the 68th anniversary of the US atomic bombing of the city, as anti-nuclear sentiment runs high in Japan.

The annual commemoration came as radioactive water leaks from the crippled Fukushima nuclear plant have rekindled fears about its precarious state, and underscored broader worries over atomic power following Japan's 2011 nuclear crisis.

In Hiroshima, ageing survivors, relatives, government officials and foreign delegates observed a moment of silence at 8:15 am local time, when the detonation turned the western Japanese city into an inferno.

"We offer heartfelt consolation to the souls of the atomic bomb victims by pledging to do everything in our power to eliminate the absolute evil of nuclear weapons and achieve a peaceful world," Hiroshima Mayor Kazumi Matsui told the ceremony.

Prayers continued through the day as Buddhist monks, visiting students and others formed a queue in front of the cenotaph at Peace Memorial Park.

Anti-nuclear rallies were being held across the city with a candlelight ceremony planned for the evening near ground zero.

"We Japanese are the only nuclear-bombed nation," Prime Minister Shinzo Abe told the ceremony.

"We are responsible for realising a world without nuclear weapons and we have a duty to convey their inhumanity to the next generations."

UN Secretary General Ban Ki-moon said in a statement that "we are united in countering the erroneous view that security is achieved through the pursuit of military dominance and threats of mutual annihilation".

"Our memories are long. We know this path is a dead end," he added.

An American B-29 bomber named Enola Gay dropped an atomic bomb on Hiroshima on August 6, 1945, in one of the final chapters of World War II. It had killed an estimated 140,000 by December that year.

Three days later, the port city of Nagasaki was also bombed, killing an estimated 70,000 people.

Historians have long been at odds over whether the twin attacks brought a speedier end to the war by forcing Japan's surrender and preventing many more casualties in a planned land invasion.

Also Tuesday Japanese officials unveiled Tokyo's biggest naval ship since World War II, as the government moves to strengthen its self-defence forces. Tokyo said the timing of an annual peace ceremony and the helicopter carrier unveiling was coincidental.

Many atomic bomb survivors, known as "hibakusha", oppose both military and civilian use of nuclear power, pointing to the tens of thousands who were killed instantly in the Hiroshima blast and the many more who later died from radiation sickness and cancer.

Anti-nuclear sentiment flared in Japan after an earthquake-sparked tsunami left some 19,000 dead or missing and knocked out cooling systems at the Fukushima Daiichi nuclear plant two years ago.

None of those deaths were directly attributed to the nuclear crisis. But reactor meltdowns spread radiation over a large area and forced thousands to leave their homes in the worst atomic disaster since Chernobyl in 1986.

Concerns about Fukushima have remained high since the accident. In recent weeks, the plant's operator Tokyo Electric Power admitted for the first time that radioactive water had leaked into the ocean as it struggles to contain the toxic buildup.

"Eastern Japan is still suffering the aftermath of the great earthquake and the nuclear accident," Hiroshima's mayor said Tuesday.

"The people of Hiroshima know well the ordeal of recovery. We urge the national government to rapidly develop and implement a responsible energy policy that places top priority on peoples' safety and livelihood."

Abe's administration has advocated restarting Japan's switched-off nuclear reactors if their safety can be assured, a plan opposed by many.

His conservative Liberal Democratic Party has also said it wants to upgrade Japan's self-defence forces into a full-fledged military, which would mean overhauling a pacifist constitution imposed on the country by the US and its allies after WWII.