You're able to restrict ChillBot to accept commands in certain text channels and also only join certain voice channels.
If you would like ChillBot to NOT join a certain Voice Channel or read commands from certain Text Channels. Deny ❌"View Channel" for the ChillBot role. If a bot can not view a channel it can not interact with it.
If the bot has any role that gives it ✅Green Tick in Channel Permissions to View Channel, the bot will not be restricted as it overrides ✅> ❌. For more info read discord article about permissions