You can turn on Case Requests which means that rather than being automatically added to a case, you now have to accept or decline each request to be added to a case
Steps
Turn on Case Requests by going to your Edit Settings page, navigating to the Email Notifications tab and ticking the “I must accept an invitation” tickbox
Press Save Settings
Related pages
Get Notified of an incoming invitation (request) to join someone else’s Case