You are making an assumption the old router failed because of overheating. Unless you have significant proof that is what caused it, you may be concerned for nothing.
Consumer routers are designed to withstand certain operating temperatures (since you didn't give any specific model info, I could not look up the operating temperatures). They don't typically have fans because they don't generate a lot of heat and they are designed passively release heat via the builtin heat-sinks.
Using a small fan to create some airflow will not hurt the router and will give you some peace of mind. The only issue, you may need to blow the dust out the router occasionally due to the introduced airflow.