Stay in Kenya, its far better by all means.

Tanzania its a Muslim country.

But if you are curious just take the bus from mombada to several town "near" the border. English is not widely spoken...