Spanish activists on Monday removed part of a controversial artificial reef which was built in disputed waters near Gibraltar despite objections from Madrid.

The two-tonne concrete block was one of 70 that Gibraltar dropped into the sea in July 2013 saying it was creating an artificial reef that would boost fish populations.

Madrid complained the barrier prevents Spanish fishing boats that trawl for shellfish from operating in the area, and in response, introduced stringent border checks on its border with Gibraltar.

The checks caused a diplomatic row and lead to lengthy delays for motorists trying to enter the tiny British outpost on the southern tip of Spain.

A nationalist group, the Foundation for the Defence of the Spanish Nation, said four divers and a fishing boat had removed the concrete block from the reef before dawn and taken it to the Spanish port of San Roque.

The group -- which accuses Spain's conservative government of not doing enough to remove the artificial reef -- published photos of the haul on its website.

"Four divers and a small fishing boat were enough," the president of the group, Santiago Abascal, wrote in a Twitter message alongside a photo of himself with the divers.

Spanish police have since seized the block from the activists, he added.

The removal of part of the reef comes as Britain's minister for Europe, David Lidington, is in Gibraltar for a two day visit.

Lidington toured the Gibraltar's border with Spain with Gibraltar Chief Minister Fabian Picardo but did not speak to reporters.

"The removal of a concrete block is an individual act of vandalism. It is not a violation of British sovereignty," said a spokesman for Britain's Foreign Office, who added that it is up to Gibraltan police to uphold the law in the British territory.

The mayor of Algeciras, the port which lies across the bay from Gibraltar, condemned the move by the activists, saying it could have posed a risk to boats sailing in the area.

"Spain is seeking to settle the conflict as serious and sensible countries should, through the courts of the European Union," Jose Ignacio Landaluce, a member of Spain's ruling Popular Party, told reporters.

Spain ceded Gibraltar to Britain in 1713 but has long argued that it should be returned to Spanish sovereignty. London says it will not do so against the wishes of Gibraltarians, who are staunchly pro-British.