Pressure is growing on the town hall after it approved plans for new homes which don't include a promised community centre.
Newham Council's bid to build 65 flats and a nursery at the former site of The Upton Centre in Upton Park was given a green light in February.
But past users of the community centre in Claude Road say Newham has rowed back on a pledge made during Sir Robin Wales's administration to provide a space at the redeveloped site.
Community groups had to move out of the centre which was demolished to make way for development in 2016.
Simon Davis, of Claude Road, said: "We were assured the vibrant community hub that was lost would be replaced.
"This is another example of residents' concerns being ignored."
Miraj Patel, who advises the Upton Centre Association, said it reserves the right to take whatever lawful and proper means there are to challenge the council's decision.
But he added it doesn't think it will be necessary as Newham's mayor, Rokhsana Fiaz, considers faith and community to be "very important".
A council spokesperson said Newham has been active in finding alternative space for most groups.
Ms Fiaz said affordable homes and opportunities for young people are key concerns shared with her by residents and the new homes show Newham is acting on people's priorities.
Varsha Mistry, secretary of the Milan Mahila Group which used the centre, said: "If you made those promises, then the question is whether there is an obligation to see them through.
"[The council] might not be able to deliver on the same site, but I would hope they would do their best to find a solution which helps them build houses and delivers on the promises given by Sir Robin Wales."
Ms Mistry said the group is a lifeline for its elderly members who are from the borough's Indian community, but membership has dropped from 60 to 38, in part due to the centre's loss.
She added an alternative site, Valetta Grove, is too far from public transport.
However, Ms Fiaz said she felt confident Valetta Grove would be well used, adding it is next to Plaistow Station and a 13 minute walk from the old centre.
"When I became mayor in May 2018, I pledged to make sure community facilities are accessible and inclusive to all," she said.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here