It is right to be concerned that people who have not paid into the welfare system (regardless of where they come from) are abusing the system. If people can come to the UK and start drawing state benefits without working and becoming a burden on the state, then that is not fair on those who are paying for it.
Thankfully, the EU does not allow this.
EU freedom of movement rules allow EU citizens to come to the UK for up to 3 months. After this, they must have a job, or other income so that they are NOT a burden on the state.
Freedom of movement does not entitle migrants to claim out of work benefits.
Any EU migrants who become a burden on the state can be sent back to their home nation.
All of this is enshrined in the rules governing freedom of movement.
But, you may have seen several newspaper articles about EU migrants who are claiming benefits and have apparently become a burden on the state. Unfortunately, the UK Government isn't completely efficient at identifying people who have become a burden and sending them home. This is the UK's fault, not the EU's. Most other EU member states are much more efficient.
Whilst you are right to be concerned about abuses of the system, the blame lies with the UK Government, that has been more focussed on encouraging immigration to sustain the growth of the economy, than adequately managing the country's administration.