How can I decrease table row heights in this case?
You need to check your paragraph formats:
Open the "Paragraph" dialog
Select "Indents and Spacing" tab
In the "Spacing" section make sure "Before" and "After" are set to "0 pt"
See below for instructions and (Windows) screenshots.
Paragraph Formatting and Creating Lists
You can control paragraph behavior and appearance using the “Paragraph” tab. This tab has several notable features including increase/decrease indent, line spacing, borders, and more. There are many more options than are first apparent. To access that click on the icon at the bottom corner of the paragraph tab.
The “Paragraph” dialog will give you further, more fine-grained control. You can affect indenting, line spacing, as well as line and page breaks.