An interesting trip from Kigali is the church in Nyamata, where several thousand people were massacred during the genocide. You can still see the blood stains on the walls. The church is now a memorial site and there is always someone to give you explanations of what happened. The road is in bad condition though, so despite being only 35km or so away from Kigali it can take up to 1.5 hours one way to get there by car/taxi (around US$50 for the return trip).
It is now possible to visit the site of Dr Dian Fossey’s grave in the Parc Nacional des Volcans and do other treks in addition to visiting the mountain gorillas. Security is very tight and groups are always accompanied by soldiers. Note that you are not allowed to take photos of them, not even when they are standing in the background.
You are not allowed to take photos of anything ‘official’ (e.g. military buildings, soldiers). If in doubt, it is better to ask permission before taking a photo.