There is one major issue and possibly one minor issue with your xterm call. First - You don't want the & to background the script - it is very likely doing exactly what you asked it to do and runs in the background (effectively invisible - you can see it with ps or jobs in the shell that you launched it from; ps -A from anywhere, etc.).
My read of your situation is that you are trying to spawn a new xterm instance and have this script execute in it; and it will do not that in the background (at least visibly).
Second, you should provide the full path to my_script.sh, use it's full path, and enclose the argument to the -e flag entirely in quotes, e.g. "/home/you/my_script.sh"
That may not be causing you an issue here; I'm not positive. It will cause you an issue at some point if you write similar scripts in the future, as without quotes you could be the victim of inadvertent bash expansion.
Last note - when calling a bash script from a bash shell, the #!/bin/bash is actually completely unnecessary. It's good habit, and it makes it more portable - someone else will jump on this and say '#!/usr/bin/env bash is MORE portable',and yes, it is. But you don't need either for it to work, assuming you call the script from bash (but you should always have it anyways).