I fixed this by generating a new certificate without the -batch parameter. This means the certificate asked for "common name" when generating it.
I put the logstash server dns as the common name, replaced the old certificates with the new one, and it worked. Here is the command I used to generate the new certificate:
openssl req -x509 -nodes -newkey rsa:2048 -keyout logstash-forwarder.key -out logstash-forwarder.crt