...it just works the same, it doesn't include any context lines.
Because there is no context! You've explicitly told Get-Content to grab 1 last line with -Last 1, so Select-String receives only one line.
Can anyone tell me why and what I could do to get context lines like this?
Sure, all you have to do, is increase number of lines for the Last parameter. Assuming that you match 1 line in regex and want 1 line before and 2 after it (-Context 1,2) then total amount of lines would be 1 + 1 + 2 = 4:
Get-Content -Path 'xxx.log' -Tail 4 | Select-String -Pattern 'Exception' -SimpleMatch -Context 1,2 | ForEach-Object { $_.Context.PreContext $_.Line $_.Context.PostContext }