Minister for Railways Development Joe Ghartey has warned the ministry will not allow the activities of encroachers to frustrate their effort to put the railway sector back on track.