Residents confronted Select Board Chair Brian Butler at a meeting where some called for his resignation while others defended the controversial post.