how can that be since the pop instruction itself is 1 byte large?
It's encoded to different bytes depending on which register it uses. pop ebx
is 0x5B, pop esi
is 0x5E. There are many websites and manuals documenting the x86 instruction encoding.