Police were called to take the topless protesters in to custody. The protesters accused the International Olympic Committee of supporting 'bloody' Islamist regimes.
Pictures showed officers holding women on the ground and covering their chests.One woman screamed as officers held her. One of the women carried the message 'No Sharia' on her bare chest.
The demonstration near Tower Bridge, described as an 'Islamic marathon' by the protesters, was organised by the Ukrainian feminist group Femen.
Femen, a group based in Kiev, was founded in 2008.
The organisation has become internationally known for organizing topless protests against sex tourists, international marriage agencies, sexism and other social, national and international ills.
Its stated goals include developing 'leadership, intellectual and moral qualities of the young women in Ukraine' and to 'build up the image of Ukraine, the country with great opportunities for women'.
In a statement published on its Facebook page, the protest group said a number of its French members had been arrested.
It called on the IOC to 'condemn violence towards women from Islamist states'.
It said: 'Femen requires the states that apply the laws of sharia be shut out of the Olympic games, because of the inhumanity of these laws, which are in total contradiction with Olympic principles that are peace and philanthropy.
'Femen accuses these states of simulating democratic changes regarding the position of women.
'With the support of the IOC, these Islamist governments use the participation of women in the Olympic games to hide thousands of victims and dead.
'If the IOC keeps flirting with radical Islam, new Olympic disciplines, such as stoning or speed raping will be added to the competition.'
A spokesman for the Metropolitan Police said: 'At approximately 11.30am today police attended the site of a demonstration outside City Hall.
'Four topless female protesters were arrested under section five of the Public Order Act for disorderly behaviour. They are currently in custody.'
The protest was staged less than three months after Femen activists snatched football's European Championship trophy on two occasions.
They grabbed the cup in May while it was on display in a central square in Dnipropetrovsk, south-east Ukraine.
One activist took off her top to reveal an offensive slogan before picking up the trophy.
As security guards moved in on her, another activist, standing next to the cup, stripped off to the waist to display the same slogan.
Activists staged a similar protest in the Ukrainian capital Kiev earlier that month.
In February the protest group clashed with Italian police during Milan Fashion Week after a demonstration against the use of anorexic models.
The topless protesters wore jeans and had handwritten slogans such as 'Fashion = Fascism' and 'Anorexia' scrawled across their chests.
A similar protest was held in January in the Swiss resort of Davos in January, where the World Economic Forum held its annual meeting.