There is any easy way to solve the problem in general that has nothing to do with emacs at all and will work for any program that can choose between running in the console vs X:
unset DISPLAY
Of course you may not want to put that in your configuration file to be applied globally to all your shell sessions, so if you want it to apply to only emacs, then either call it from the command line like this:
DISPLAY= emacs
note the space!!! if you leave the space out it means you're setting the DISPLAY to emacs instead of setting DISPLAY to nothing... this command is a shorthand for:
DISPLAY=; emacs
So either use the above from the command line(s) or put that in a wrapper script that would look something like this:
#!/bin/bash
unset DISPLAY
exec emacs
I recommend the exec there because it will replace your wrapper script with emacs; to see the difference between the two you can run:
pstree -p